NVIDIA's new Bolt model is built to run voice agents and game characters on your own device

A small model for talking machines

As marktechpost.com reported, NVIDIA has released Nemotron 3.5 Bolt E4B 4GB, an open model built for conversation. The targets are voice assistants, characters in video games and the chat layer of robots. All of them are meant to run on the device itself, using NVIDIA RTX graphics cards, Jetson boards and its Spark machines.

Bolt is a small language model, meaning a text-generating AI with far fewer settings (parameters) than the giants behind chatbots. It has 8B parameters in total, but only 4B are active for any one reply. It also has a 64K context window, which is the amount of text it can keep in view at once.

NVIDIA also describes it as the third generation of small models for its ACE game-character technology. In a demo, a Reachy Mini robot ran on a Jetson Orin Nano 8GB board. Another NVIDIA model, Cosmos 3 Edge, handled vision, while Bolt produced spoken replies and triggered tools.

Trained without the usual first step

Bolt starts from Google's Gemma 4 E4B, an existing open model. NVIDIA then reshaped its behavior using reinforcement learning, where a model improves by being scored on its answers rather than copying examples. Usually a team first teaches by example, a step called supervised fine-tuning. NVIDIA skipped it, saying it made tool calling better but made Gemma's chat worse and wordier.

Training used two kinds of scoring. One checks answers that can be verified, and the other uses human-style preference signals. Every training sample was run with the model's step-by-step reasoning both on and off. Each batch held two non-reasoning samples for every reasoning one, and penalties for long answers kept replies from bloating. NVIDIA says the datasets and full recipe are public.

The model also has a few practical controls. A flag called conversational_mode asks for short replies that sound natural when spoken. A thinking budget caps how long the model can reason, which limits delay. It also supports multi-turn tool use, where the model calls outside software across a conversation.

What the scores say, and what they don't

NVIDIA grouped 21 benchmarks into five areas and says Bolt came first on 15 of them in reasoning mode. The rivals were Gemma 4 E4B, Qwen3.5-4B and NVIDIA's earlier Nemotron 3 Nano 4B. The biggest gap was IHEval, which tests whether a model follows a ranking of instructions. Bolt scored 84.8, against 67.0 for Gemma and 60.6 for Qwen.

NVIDIA also claims average scores about 40% higher than Nemotron 3 Nano 4B and up to 80% fewer output tokens in reasoning comparisons. Separately, Scale Labs ranked Bolt first among small models on MultiChallenge at 56.03%, according to an X post.

The weak spots matter just as much. On Tau2-Bench, which tests multi-turn customer-service tasks, Bolt scored 55.1 while Qwen3.5-4B scored 75.1. Qwen also led on a 64K-token retrieval test, 95.1 to 87.0, and on BFCL-v3, a test of picking the right tools. Every score comes from NVIDIA's own testing setup, so independent checks may differ.

Reading the '4GB' label carefully

The name suggests a small download, but that is not quite right. The 4GB describes the memory the model is meant to need while running. The compressed file (Q4_K_M, a format that stores numbers at lower precision to save space) is 6.18 GB on Hugging Face. NVIDIA's tests on RTX cards used between 3.1 and 4.3 GB of video memory.

Speed depends heavily on hardware. On an RTX 5070 with 12GB, NVIDIA measured 3.5 GB of memory, 548 ms to the first token and 133.4 tokens per second. On the Jetson Orin Nano 8GB, the first token took 5,785 ms and generation ran at 15.5 tokens per second. These figures used 3K tokens of input, so a long prompt on small hardware will feel slow.

NVIDIA also ships a helper model, DFlash, that guesses several words ahead for the main model to check. On the RTX 5070 that reached 272.6 tokens per second, versus 225.9 for Gemma 4 E4B with its own helper, a 1.21x edge.

Who can use it and on what terms

Bolt is on Hugging Face, Ollama, LM Studio, Unsloth and Jetson AI Lab. It also comes as an NVIDIA NIM, a ready-to-run package, and works with the In-Game Inferencing SDK. A free API is available on build.nvidia.com for trying it without hardware.

The license is OpenMDW-1.1, according to the comparison table in the report. Gemma 4 E4B and Qwen3.5-4B use Apache 2.0, so read the terms yourself before building a commercial product. Bolt's context is also shorter than Gemma's 128K or Qwen's 262K, which matters for long conversations or documents.

The natural audience is developers and hobbyists with a recent gaming PC who want a prototype voice helper or game character. Students get a model small enough to run and study at home.