Technology & Research · 2026-10-13 · Team LATENT

ColGrep-Agent delegates code search to a local 2B model

How a local 2B model searches for a larger coding agent, what the token savings trade off against time and accuracy, and how this relates to local LLM configurations.

When an AI fixes a bug, it may search and read files repeatedly just to locate the code. If a large model handles every step, candidate code and search history accumulate in its API input. ColGrep-Agent, released by LightOn on October 12, 2026, delegates that stage to a local 2B model.

The small model locates code relevant to a question. Planning, editing, testing and accepting changes remain with the parent agent. Compared with moving a whole large LLM onto a personal machine, this gives local inference a narrowly defined responsibility.

Key takeaways

ColGrep-Agent delegates code exploration to a local model with two billion parameters. It returns relevant files, lines, and code to the higher-level agent.
In LightOn’s evaluation of 52 questions across 13 repositories, API token use fell by 34.3%. However, elapsed time increased from 13.0 to 21.6 seconds, while accuracy changed from 83.3% to 80.8%.
Running code exploration locally does not make the higher-level agent local. Adoption depends on which code is returned to a cloud model and how the token savings compare with acceptable latency and accuracy.

Large-model and local-2B responsibilities

Send a question and receive relevant code

1 Parent agent

Read the issue → delegate a search question. Own planning and edits.

2 Local ColGrep-Agent

Search with ColGrep → read candidates → follow references. Repeat locally.

3 Results returned upstream

Paths + line ranges + code → the parent checks them, then edits and tests.

Figure 1. Conceptual flow based on the official implementation. Returning results to a cloud parent does not make the whole workflow local.

LightOn announcement and Claude Code evaluation

What changes when search is delegated

ColGrep searches code by meaning, using NextPlaid as its retrieval engine. ColGrep-Agent-2B is the model that calls search and reads candidates. The colgrep --agent command combines that model, the search tool and read-only terminal operations.

For a question such as “Where is the retry count configured?”, the agent can read candidates and find other files that refer to the setting. It returns inspectable code locations, not only an explanation. This can reduce the parent model’s need to repeat the exploration from scratch.

The model card describes search, a read-only terminal and a finish tool within a ten-turn budget. Its SWE-bench Lite and other evaluations are adapted into file and line localization. The benchmark names should not be read as evidence of completed bug fixes.

Model card: tools and localization evaluation

Read tokens, time and accuracy together

The integration evaluation uses Claude Code with Sonnet 5.5 on 52 questions from 13 repositories. Each question runs three times under four search configurations: 624 sessions, not 624 independent tasks.

Setup	API token reduction	Correct answers	Time per question
Built-in Grep	Baseline	83.3%	13.0 s
Direct ColGrep	24.6%	79.5%	15.0 s
Delegate to ColGrep-Agent	34.3%	80.8%	21.6 s
Balanced combination	21.2%	83.3%	17.7 s

The reduction covers Claude Code API input and output tokens, excluding local-2B generation. The 95% confidence interval for the 34.3% reduction is 28–40%. It is consistent with keeping exploratory material out of repeated parent-model calls.

Savings alongside added time

Reported Grep versus ColGrep-Agent observations

API tokens

About 142k → 93k; a 34.3% reduction. Local computation is separate.

Elapsed time

13.0 s → 21.6 s. This is not a measured speedup in search.

Correct answers

83.3% → 80.8%. No statistically significant difference does not prove equivalence.

Figure 2. 52 questions, 13 repositories, three runs per question. This does not compare full implementation, review and testing workflows.

The authors report no statistically significant accuracy difference but explicitly say a meaningful decline cannot be ruled out. Cache-aware API cost fell from 7.3 to 4.4 cents per question. That is separate from total cost including local compute, waiting, review and recovery from failed searches.

Numbers and methodology: announcement sections 3.1–3.5

Memory needs and what stays local

The default colgrep-agent-2B-Q8_0.gguf is about 2.5 GB and is downloaded once. File size differs from peak runtime memory. The official blog reports about 20 seconds per question and 3–4 GB peak on an M3 Pro with Metal. These figures do not establish requirements for Windows GPUs or CPU-only execution.

Inference uses an available GPU and otherwise falls back to CPU. The blog documents Metal on Apple Silicon and Vulkan for other supported GPUs. Indexes and other applications also consume resources, so available memory cannot be judged from the 2.5 GB weight file alone.

Search and 2B inference can run locally. But returned code becomes input to a cloud parent when one is used. For confidential repositories, the location of retrieval and the destination of its results need separate checks.

Default model and M3 Pro resource conditions

Integrating with Codex and Claude Code

Official installation paths cover macOS, Linux and Windows. After installing ColGrep, the workflow is model setup, indexing, a standalone query, then agent integration. The commands below are documented examples; LATENT did not download the model or run its benchmarks for this article.

colgrep --install-agent
colgrep init
colgrep --agent --json "where are retries configured"
colgrep --install-codex

The first command downloads about 2.5 GB. For Claude Code, colgrep --install-claude-code installs hooks that suggest search tools rather than force their use. The repository describes full hook support in Claude Code and basic integrations for Codex and others. Their integration depth should not be assumed equal.

Official repository: installation and integration scope

Our practical local-LLM article considered which model sizes fit which memory configurations for regular use. ColGrep-Agent changes the question: can a repeated search stage move to a small model while the large parent stays in place?

Strata concerns how to handle a large MoE model’s weights; ColGrep-Agent delegates work to a separate model. It also differs from Strands Decider: Decider selects among options, whereas ColGrep-Agent executes search-and-read steps and returns supporting locations. Small size alone does not imply the same computation or latency.

Evaluate search separately from fixes

For an adoption trial, choose repository questions whose answers you can check. Mix known-symbol questions with behavior descriptions. Compare API tokens, elapsed time and missed locations between built-in and delegated search. This is a proposed trial, not a LATENT measurement.

Then separately measure test outcomes and review effort. Savings in search can be offset by rework after editing the wrong place. Evaluating it as a code-location specialist makes it easier to define its responsibility and fallback conditions.

Sources and scope

Checked against the official blog, model card and GitHub on October 13, 2026. Figures are LATENT conceptual diagrams. We did not run the model, benchmark speed or evaluate Japanese search accuracy. M3 Pro figures are not generalized to other hardware, and integration usability has not been tested here.