NVIDIA releases Bolt E4B weights for concise local conversations

NVIDIA released the weights for Nemotron 3.5 Bolt E4B 4GB on October 23, bringing its small conversational model to local deployments. The model card records the release on Hugging Face and NVIDIA’s build platform on that date.

The public repository contains a Q4_K_M quantized GGUF file, listed at 6.18GB. That is the downloadable artifact’s size. NVIDIA separately reports GPU memory during inference; the “4GB” in the product name is not the size shown for this file.

On an RTX 5070 with 12GB of memory, NVIDIA measured peak GPU usage of 3.5GB for 3,000 input tokens and 50 output tokens. With 32,000 cached tokens, 2,000 new input tokens and 256 output tokens, usage reached 4.3GB. Both configurations used Q4_K_M, llama.cpp with CUDA, and one request at a time.

These are NVIDIA’s AIPerf measurements for selected workloads. The approximately 4GB target does not establish compatibility with every 4GB device, or the total memory needed by a complete voice application. The tested GPU itself had 12GB.

Built on Gemma 4 E4B-it, the model has eight billion total parameters and four billion effective parameters. Its card specifies a context of up to 64,000 tokens. Reasoning can be enabled or disabled independently of conversational mode, which also works with tool calls. A runtime thinking budget can cut reasoning off mid-sentence, according to the card.

The technical report explains that conversational mode adds a system instruction for concise, lightly formatted answers. When an assistant message combines text and tool calls, its template places the text first, allowing an acknowledgement or explanation before a tool executes.

Training involved a trade-off: supervised fine-tuning improved tool use but worsened conversation and verbosity. NVIDIA therefore omitted that stage from the released model’s post-training.

English-only post-training also affected multilingual results. In the report’s BF16 reasoning evaluation, Korean MMLU-ProX scored 64.51, down 1.77 percentage points from the base model. This is a different precision setting from the distributed Q4_K_M artifact.

NVIDIA’s explainability document flags sensitivity to instructions in untrusted inputs, including prompt injection. It calls for guardrails and data filtering. In agent systems without effective controls, the company identifies possible consequences including information access beyond intended permissions and remote code execution.

The model uses OpenMDW 1.1. That agreement grants permission free of charge subject to its terms. Redistribution of model materials must retain the agreement and applicable copyright and origin notices. It adds no obligations for using, modifying or sharing generated outputs. The license provides the materials without an accuracy warranty.