Nemotron 3.5 Bolt versus 3 Super: compression and the conditions behind 24GB

On October 15, 2026, NVIDIA announced Nemotron 3.5 Bolt. It brings large-model capabilities toward local deployment by changing the model structure, weight precision and generation method together. The “24GB” in its name expresses that design target. NVIDIA announcement

Comparing Bolt with the earlier Nemotron 3 Super shows what became smaller and what remained. Its direct parent, however, is the newly announced 3.5 Super VL. Performance comparisons with 3 Super and compression from the parent answer different questions.

Separate 3 Super from the direct parent

Announced on March 11, 3 Super combines efficient sequence processing through Mamba with attention for retrieving contextual information. Its mixture of experts (MoE) selects only some expert networks for each input. LatentMoE reduces the dimensionality of representations sent to experts, limiting computation. This shared foundation makes it useful for examining which costs Bolt reduces. 3 Super architecture

Smaller alternatives include 3 Nano 30B-A3B and 3.5 Lightning 30B-A3B. Nano has 30B total and actually 3.5B active parameters; Lightning has 30B and 3B. They provide a secondary comparison for model scale. Bolt, at 74B total and 9B active, is larger than both. “24GB” does not mean fewer parameters than a 30B model.

Change structure, precision and generation

Structural compression uses Star Elastic to choose expert counts and intermediate widths for a given budget. Bolt also reduces the selected experts per token from 22 to 9. Further training restores capabilities including vision and tool use. Application to Bolt

Storage precision varies by component. FC1 and FC2 in routed and shared experts use IQ2_XXS at 2.0625 bits per element; Mamba linear layers use NVFP4, while attention and embeddings use Q8_0. The entire model is not uniformly 2-bit. Bolt model card

Quantization-aware distillation (QAD) trains a quantized student to match a teacher’s output distribution, compensating for changes introduced by lower precision. QAD was already used with Nano. Bolt combines it with structural compression and roughly 2-bit experts. Recovery training does not guarantee that every capability matches the parent.

LiLiCorr changes generation. Tokens drafted in parallel can look plausible individually yet fail to form a coherent sequence. A lightweight network scores compatibility between adjacent candidates to construct sequences the main model is more likely to accept. The main model still verifies them; not every drafted token becomes output. Earlier 3 Super already used MTP for speculative decoding, so speculation itself is not new to Bolt.

What the 24GB budget includes

NVIDIA’s 23.83GB estimate assumes text inference, batch size 1, a context length of 131,072 and a Mamba prefill chunk size of 8,192. Table 2 and §8.2 of the technical report give this breakdown.

The total is reproduced as reported; adding the rounded subtotals differs by 0.01GB. The 1.68GB vision encoder and projector are excluded. This is not total device memory including the operating system and other applications. Image input and longer contexts require checks under their own deployment conditions.

The earlier 3 Super is the baseline for agent capabilities; Nano provides a smaller local-model reference for storage. Published capacity figures describe different quantities. The table below records what each source establishes, rather than ranking memory requirements.

Context limits also matter. Model cards specify up to 1M tokens for 3 Super and 3 Nano, versus 256K for Bolt. These specification limits differ from the lengths used in the deployment recipe and memory estimate. Bolt’s 23.83GB figure cannot be applied to a full 256K context.

Where Bolt leads and trails 3 Super

For a comparison with 3 Super, use the values in Table 7 of the same report. These are NVIDIA’s evaluations: BF16 for 3 Super and the 24GB quantized variant for Bolt. The published scores below belong to different evaluations and should not be summed or averaged across rows.

Bolt scores higher on Terminal-Bench and BrowseComp; the BrowseComp evaluation uses the Exa search API. The earlier 3 Super scores higher on AA-Omniscience accuracy and IFBench instruction following. These results do not imply universal improvements in knowledge or instruction adherence.

Generation, training, architecture and storage precision all change between these models. The difference cannot be attributed to quantization alone. Start with evaluations close to the intended workload, then check representative inputs in the actual environment.

Separate generation speed from task time

NVIDIA measured speed with llama.cpp on RTX Spark in a Surface Laptop Ultra with 128GB. Bolt plus LiLiCorr achieved 83.99 tokens/sec in SPEED-Bench Coding and a 600.80-second replay duration in AA-AgentPerf Local. These are not speed measurements on a 24GB GPU. Measurement configuration and comparison table

Tokens per second measures generation throughput. AA-AgentPerf Local replays recorded agent trajectories and skips tool execution time by default. Its 600.80 seconds cannot be read as the time needed to autonomously solve a new task correctly. NVIDIA’s table also does not specify its tool-mode details.

Real tasks also involve input processing, tool latency and retries after failures. The table has no 3 Super speed result on the same device, so it cannot establish a speedup over that earlier generation.

What to check for local deployment

Bolt’s model card lists Linux and NVIDIA Blackwell SM120 and SM121 for its vLLM integration. That is a different execution path from the Windows llama.cpp speed measurement. Having at least 24GB does not by itself establish support for the same format and features on every GPU or runtime. Software Integration and Quick Start

Check the quantization format, supported inference implementation, context length, image requirements and concurrency first. The official model card provides GGUF / llama.cpp and ModelOpt / vLLM paths. Loading the weights is only one requirement; the needed features and processing time matter too.

The practical local LLM article examines how local models connect to useful work. The TensorFold article explains an inference runtime for trained models and how the main model verifies speculative candidates. Bolt also changes the model’s own structure and storage precision, then adds recovery training. Verifying speculative tokens and preserving capabilities from before compression are separate questions.

Bolt E4B for a 4GB budget is scheduled for October 23. As of this article’s verification date, October 16, it should not be treated as already available alongside the 24GB variant. Figures in this article come from primary sources; LATENT has not run the model or measured it on hardware. Availability announcement