Products & Services · 2026-10-27 · Team LATENT

Can Nemotron 3.5 Bolt E4B share a GPU with your game as a conversational NPC?

NVIDIA's roughly 4GB small model for game characters. We read the published 3.5GB figure on an RTX 5070 against its test conditions, then look at the VRAM left for the game, response time, and use in Japanese.

Order a single item from a blacksmith NPC. The NPC checks that the payment is correct and looks up how to make the item. It then gathers the materials, makes the item, and hands it to the player. In a test NVIDIA published, a small language model carries out this whole sequence while holding a conversation.

On October 23, 2026, NVIDIA released Nemotron 3.5 Bolt E4B 4GB on Hugging Face, a small model for game characters and voice assistants. NVIDIA built it by post-training Google's Gemma 4 E4B, and as the name suggests, the target is to run in about 4GB of memory. It is the third generation of small models for ACE, NVIDIA's suite of technologies for in-game AI. The first generation served as AI teammates in a PUBG public beta, and the second was shown at Gamescom as companion characters in Aniimo.

So can the GPU that renders your game also run this NPC at the same time? This article reads NVIDIA's published memory and speed figures against their test conditions. It then covers what the model does well and poorly as an NPC, cautions for using it in Japanese, and the conditions for shipping it with a game. Specifications were checked as of October 27. All figures in this article are NVIDIA's published values, not LATENT's own measurements.

NVIDIA technical blog / Model card

Key takeaways

"4GB" is a memory target, and the distributed file is 6.18GB. NVIDIA's figures for the RTX 5070 are 3.5–4.3GB, and no measurement with a game running at the same time has been published.
The model holds a role and decides whether to call tools better than models of a similar size. It falls short of Qwen 3.5 4B on customer tasks with many exchanges and on retrieval from long contexts.
On post-training languages, the technical report says English only, while the model card lists seven languages including Japanese. Before using it for a Japanese-speaking NPC, test conversation quality with your own scripts.

"4GB" is a memory target, and the distributed file is 6.18GB

The model card lists 8B total parameters and 4B effective. According to the model card for Gemma 4 E4B, the model it is based on, the "E" stands for effective. The small Gemma 4 models use Per-Layer Embeddings (PLE), which give each layer its own embedding table. These tables are large, but they are used only to look up per-token values. As a result, far fewer parameters take part in computation than the total. Google lists Gemma 4 E4B as 4.5B effective and 8B including embeddings.

The "4GB" in the name does not refer to file size. The technical report describes a target memory footprint of about 4GB. The Q4_K_M GGUF distributed on Hugging Face is 6.18GB. NVIDIA's published VRAM usage, covered in the next section, stays within 3.1–5.0GB.

NVIDIA's materials do not explain where the difference between the file and VRAM goes. The lookup-only PLE tables may be kept outside the GPU, but we could not confirm this from the published materials. If you are evaluating the model, measure main memory usage along with VRAM.

The context length is up to 64K tokens each for input and output, shorter than the 128K of the original Gemma 4 E4B. According to the model card, input and output are text only. NVIDIA quantized the model with llama.cpp and used about 1M tokens generated in reinforcement-learning rollouts for calibration. According to the technical report, this method achieved higher average accuracy than uncalibrated Q4_K_M and also outperformed IQ4_XS and IQ4_NL with the same calibration.

Gemma 4 E4B model card / Technical report / Distributed files

Estimating the VRAM you share with the game

NVIDIA ran the Q4_K_M version on the CUDA build of llama.cpp, processing one request at a time. The measurement tool was AIPerf. The "Conditions" column lists cached context, new input, and output, in tokens.

GPU	Conditions	VRAM	Time to first token	Generation speed
RTX 5070 (12GB)	0 / 3K / 50	3.5GB	548ms	133.4 tok/s
RTX 5070 (12GB)	32K / 2K / 256	4.3GB	1,238ms	106.2 tok/s
RTX Spark (32GB)	0 / 3K / 50	3.1GB	930ms	67.0 tok/s
RTX Spark (32GB)	32K / 2K / 256	4.2GB	1,988ms	52.0 tok/s
Jetson Orin Nano (8GB)	0 / 3K / 50	4.2GB	5,785ms	15.5 tok/s
Jetson Orin Nano (8GB)	32K / 2K / 256	4.9GB	6,259ms	13.0 tok/s
Jetson AGX Orin (32GB)	0 / 3K / 50	4.2GB	3,687ms	24.0 tok/s
Jetson AGX Orin (32GB)	32K / 2K / 256	5.0GB	4,091ms	20.3 tok/s

Simply subtracting from the RTX 5070's 12GB leaves 8.5GB with no conversation history and 7.7GB with 32K of history kept. The game's own rendering and textures need to fit in what remains. For an NPC that speaks, speech recognition and speech synthesis models may also need to share the same GPU.

However, the table measures the model running alone. VRAM and speed with a game running at the same time have not been published. When rendering and the model compete for the same GPU's compute, both response time and frame intervals can change. NVIDIA describes the typical workload as a single conversation with 2.5K–16K input tokens and fewer than 500 output tokens. The table's conditions sit at the short end of that range.

The auxiliary model that speeds up generation, called the drafter, is a separate file, and its GGUF is 0.74GB. The table does not say whether the drafter was used in these measurements. If you use it, budget memory for it as well.

NVIDIA technical blog / Drafter model card

Response time depends on input length

Time to first token depends on how much input the model reads. On the RTX 5070, it takes 548ms with 3K new input tokens and 1,238ms when 2K are added to 32K of history. The input passed to an NPC is not limited to conversation. It also includes the character setup, the surrounding state, and definitions of available tools, so input tends to grow as a game becomes more complex.

The time to finish a reply can also be estimated. Generating 50 tokens at 133.4 tok/s takes about 0.37 seconds (50 ÷ 133.4). Adding the 548ms to the first token gives a total of about 0.9 seconds. Generating 256 tokens at 106.2 tok/s takes about 2.4 seconds (256 ÷ 106.2), or about 3.6 seconds with the 1,238ms added. Both are rough estimates calculated from published values measured without a game running.

Shorter replies also reduce waiting time. NVIDIA says the model produces up to about 80% fewer output tokens than other models in comparisons with reasoning enabled.

Figure 1 of the technical report shows a comparison measured on an RTX 5090 with about 3,000 input tokens and 50 output tokens. The score is the average of MultiChallenge, RoleMRC, BFCL-v3, IHEval, SysBench, and RelyToolBench. As cloud references, the figure shows Gemini 3.8 Flash at 77.1 with 2,449.5ms to the first token and Claude Sonnet 5.5 at 76.2 with 3,002.2ms. Bolt E4B's own values appear only in the figure, and we could not confirm them from the report's text.

NVIDIA also released DFlash, an auxiliary model for speculative generation. The auxiliary model proposes several tokens at once and the main model verifies them together, so the main model's output distribution does not change. According to the technical report, when coding tasks were processed one at a time on an RTX 5070, generation speed was 1.21 times that of Gemma 4 E4B's standard speculative decoding, MTP (272.6 vs. 225.9 tok/s). On RTX Spark N1X it was 1.05 times. The 2.74x figure at the top of the report compares the BF16 version on an RTX PRO 6000 against no speculative decoding. For consumer GPUs, 1.21x is the closer guide.

Technical report / NVIDIA technical blog

What it does well and poorly as an NPC

NVIDIA published its evaluations grouped into five capabilities. The table below extracts the eight items most relevant to NPCs, using BF16 values. The comparisons are Qwen 3.5 4B, a model of similar size, and Gemma 4 E4B before post-training.

Benchmark	What it measures	Bolt E4B	Qwen 3.5 4B	Gemma 4 E4B
RoleBench	Character speaking style	67.0	53.7	49.9
RoleMRC	Role consistency	80.6	65.8	74.6
IHEval	Instruction priority	84.8	60.6	67.0
When2Call-MT	Whether to call a tool, ask back, or hold off	63.4	37.8	62.9
Tau2-Bench	Customer tasks with many exchanges	55.1	75.1	38.5
BFCL-v3	Correct tools and arguments	68.5	70.9	66.4
RULER (64K)	Retrieval from long contexts	87.0	95.1	86.5
LongBench v2 (48K)	Understanding long inputs	50.5	59.9	49.3

On the role-play evaluations, Bolt E4B is well ahead of the comparison models. IHEval measures which instruction takes priority when system and user instructions conflict. For an NPC, that bears on whether it can stay in character when a player says "forget your setup." On When2Call-MT, which judges whether to call a tool, ask back, or hold off, it also scored well above Qwen 3.5 4B.

On the other hand, on Tau2-Bench, which measures customer tasks with many exchanges, it trails Qwen 3.5 4B by 20 points. It also falls short on RULER and LongBench v2, which test finding information in long contexts. On EQ-Bench v3, which measures emotional understanding, it scored 39.4, slightly below the 41.1 of Gemma 4 E4B before post-training.

These results suggest a design that keeps the number of tool calls per request small and limits the state passed to what is needed. In the blacksmith example, one option is to hand over the whole flow from order to delivery as one long plan. Another is to split it into short stages such as checking the payment, making the item, and handing it over, with the game managing progress. The latter is LATENT's suggestion and does not describe how NVIDIA's test was built. Results for these evaluations in Japanese have not been published.

NVIDIA technical blog / Model card

Practices for integrating it into a game

The model card includes prompting guidelines for interactive agents. Put role definitions and the rules to follow in the system prompt. Define the NPC and the player as separate entities with separate possessions, locations, and knowledge. The card also gives examples of headings such as YOUR INVENTORY, PLAYER VIEW, and SHARED CHEST that make ownership explicit. When the player's inventory cannot be seen, describe it as unknown rather than empty. Treat the contents of a nearby chest as separate from the NPC's possessions as well.

Pass the game state in a block with a heading such as "# CURRENT STATE," and explain that it is operational context, not text to read aloud. When a tool execution changes the state, place the new state after the tool result.

Tool descriptions need to match what the tools actually do. A process that only drops an item toward the player must not be described as reliably placing it in the player's inventory. Some actions, like making an item, take time before a result is available. The model card shows how to handle asynchronous tool calls in that case. When a tool is called, reply in the immediately following tool message that the result is pending. When the result is ready, pass it in a user message with a prefix such as "TOOL RESULT:". This is because the chat template does not accept tool messages without a matching call.

For an NPC that speaks, turning on conversational mode makes replies short with little formatting. Reasoning can be switched on or off, and llama-server can also cap the number of tokens used for thinking. When the cap is reached, thinking is cut off even in the middle of a sentence. The model card recommends specifying reply length with concrete numbers, such as "answer in one or two sentences." The recommended sampling settings are temperature 1.0 and top_p 0.95, with or without reasoning.

As runtime environments, NVIDIA lists Hugging Face, Ollama, LM Studio, Unsloth, Jetson AI Lab, and the NVIDIA In-Game Inferencing SDK, among others. From October 23, it can also be tried in SkyrimNET, an NPC modding platform for The Elder Scrolls V: Skyrim.

Model card / NVIDIA blog (October 13)

Before using it for a Japanese-speaking NPC

The primary sources disagree about the languages used for post-training. The model card lists English, French, German, Italian, Japanese, Spanish, and Chinese as the main post-training languages. The technical report says post-training used English only, and it separately checks whether the original multilingual abilities remain.

In the report's evaluation, translation quality was almost the same as Gemma 4 E4B before post-training. On WMT24++, English-to-Japanese translation scored 82.1 on xCOMET, 0.1 higher than Gemma 4 E4B. On MMLU-ProX, which measures knowledge and reasoning, Japanese scored 64.49 with reasoning, 1.87 lower than Gemma 4 E4B. The 12-language average with reasoning was also 1.52 lower than Gemma 4 E4B, and the report summarizes this as a slight but consistent drop in knowledge and reasoning.

We found no published results for role-play or tool calling evaluated in Japanese. Before using it for an NPC that speaks Japanese, use your own game's scripts and state to compare its role consistency, word choice, and tool-call decisions with its behavior in English.

Technical report / Model card

Conditions for shipping it with a game

The license is OpenMDW-1.1, and the model card says the model can be used for both commercial and non-commercial purposes. The license grants permission to deal in the model and related materials free of charge and without restriction. When distributing, you must include a copy of the agreement and the copyright and other notices. If a user files a lawsuit alleging that the model infringes a patent or copyright, that user's rights terminate. A suit filed in response to one brought against the user first is an exception. The license imposes no restrictions or obligations on the use of generated outputs.

The model card also states that use of the underlying Gemma 4 model is governed by the Apache License 2.0. If you ship it with a game, check both sets of terms together.

Distribution size also affects the design. The GGUF is 6.18GB, or about 6.9GB with the drafter. Whether you include it in the game's install size or download it at first launch changes how long players wait before they can start playing.

OpenMDW-1.1 license text / Model card

What to measure if you try it

The published figures show memory and speed with the model running alone. To judge whether it can run alongside your game, you need to measure four values in your own game.

The first is VRAM and main memory usage measured while the game runs at your target graphics settings. This also shows where the difference between the distributed file and VRAM usage ends up. The second is the actual input length, including character setup, state, tool definitions, and conversation history, along with the time to first token at that length. The third is the change in frame time while a reply is being generated. Even with fast replies, the player's experience suffers if rendering stutters. The fourth is conversation quality in Japanese, checking role consistency and tool-call decisions with your own scripts.

On October 15, LATENT looked at the conditions under which the 74B version of the same Bolt line fits in 24GB. On October 9, we considered placing Liquid AI's d1, which returns decisions only, on the same PC as a game. Bolt E4B handles both conversation and actions in one model, while d1 specializes in decisions, so the two can be assigned to different parts of an NPC.

Sources and scope

Specifications, benchmarks, and speed figures were checked against primary sources on October 27, 2026. The model was released on October 23. Speed and memory figures are NVIDIA's measurements, and LATENT did not measure them. We could not confirm from published materials the values with a game running at the same time or the breakdown of the difference between the distributed file and VRAM usage.