Nvidia releases a 4GB Nemotron model built for fast local assistants

What Nvidia published

As huggingface.co's model page shows, Nvidia has released NVIDIA-Nemotron-3.5-Bolt-E4B-4GB, a small language model (an AI text system compact enough to run on modest hardware). Our earlier note had only the listing's title to go on. The full model card is now available, so we can say more about what it is.

Nvidia did not build the model from scratch. It took Google's Gemma-4-E4B-it and trained it further, a step called post-training. The card says the result keeps Gemma's conversational strengths and improves three things: multi-turn tool use (calling outside functions across a back-and-forth chat), following instructions, and grounding (sticking to the input it is given).

The card lists the release date as 10/23/2026, on both Hugging Face and Nvidia's build.nvidia.com. The post-training data stops at May 2026.

What it is designed to do

Nvidia describes the target as fast local voice and device assistants, interactive personas, and embedded or edge interfaces. In plain terms, that means assistants that run close to the user rather than in a distant data center.

The card also stresses low memory use and low TTFT, short for time to first token, meaning how quickly the first piece of an answer appears. The stated reason is that an assistant can reply quickly and still leave room for the rest of an application or for other models running next to it.

For extra speed there is an optional companion file called a DFlash drafter. It is used for speculative decoding, a technique where a small helper model guesses upcoming words and the main model checks them. The card supplies setup commands with and without the drafter but gives no speed numbers in the text we have.

How it was trained and what the numbers mean

The card says Nvidia trained the model with reinforcement learning, where a model is rewarded for good outcomes. The method is called GRPO (Group Relative Policy Optimization). Training ran across four kinds of tasks: following instructions, multi-step tool use, multi-turn conversation, and structured output. Nvidia used its NeMo RL and NeMo Gym software, with an asynchronous setup that separates training from generating answers.

On size, the card lists 8B parameters, with 4B effective. That is the detail our earlier note told readers to check about the name's "E4B". The card does not explain the gap in the text we have, so treat "4B effective" as Nvidia's own description. The context limit is up to 64K tokens, roughly how much text the model can consider at once. Languages include English, Spanish, French, German, Italian and Japanese, among others.

Benchmark results are part of the card, but the table did not come through in the text we have, so we are not quoting any scores. Nvidia says it measured accuracy itself with its NeMo Gym harness and that its numbers may differ from other vendors' self-reported ones. Most tests were run in reasoning mode, except RoleBench, EQ-Bench v3 and RULER.

License and where it runs

The card says the model is ready for commercial or non-commercial use. It is governed by the OpenMDW License Agreement, version 1.1. Because it builds on Gemma 4, the card notes that Gemma 4's own use is governed by the Apache License, Version 2.0. Read both before you build a product on it.

The main file is in a compressed form labeled Q4_K_M, and there is a full-precision BF16 version for those who want it. Hugging Face lists ways to run it with Transformers, llama.cpp, Ollama, vLLM, SGLang, LM Studio, Jan, Docker Model Runner and Lemonade, plus agent tools such as Hermes Agent and OpenClaw. Several of these are one-line commands.

One caution: the card's supported-hardware list names Nvidia's Ampere (A100), Blackwell and Hopper (H100-80GB) chips, with Linux as the operating system. Its example server also sets a 65536-token context, and the text we have does not say how much memory that needs. The "4GB" in the name describes the file, so check real memory use on your own machine.

What to make of it

This is a release for people who want a small assistant they control. Nvidia is aiming at voice and device interfaces, where speed and a small footprint matter more than raw size. It is also another case of a large company adapting another lab's open model rather than starting fresh.

The claims about better tool use and instruction following come from Nvidia's own testing. The practical check is to run it on your own tasks and compare it with the Gemma base it came from.