# Nemotron-3-Nano 30B-A3B Chat-Aligned Base (2026.10.30) Research artifact. This model has not been properly tested or evaluated and is not necessarily secure. Use at your own risk in production settings. This is NVIDIA's pretrained [`nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16) (revision `e0ac9ee3`), after a short three-stage supervised fine-tune (SFT) that teaches it a chat, thinking and tool-call format. It is a starting point for post-training research, not a finished assistant: it has had no preference or safety tuning. It is not NVIDIA's own chat model, [`NVIDIA-Nemotron-3-Nano-30B-A3B-BF16`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16), and does not use NVIDIA's chat template. It was built for a [Marin](https://github.com/marin-community/marin) experiment on how on-policy distillation (OPD) and fine-tuning interfere across tasks in mixture-of-experts (MoE) models. The experiment gives four MoE architectures the same SFT recipe, so that later methods start from comparable chat-capable bases. The companion models are in the same collection. | | | |---|---| | Architecture | `NemotronHForCausalLM` (`nemotron_h`): 52 layers, of which 23 are Mamba-2, 23 MoE (128 routed experts, top-6, plus 1 shared) and 6 attention (32 query heads, 2 KV heads) | | Parameters | 31.6B total, about 3B active per token | | Weights | bf16, 3 safetensors shards, 63 GB | | Context | 262,144 positions; evaluated at 131,072 | | Tokenizer | the base model's, included | ## How it was trained Three stages, each one epoch of packed 32,768-token sequences, each starting from the previous stage's weights with a fresh optimizer. All stages use Adam (β 0.9/0.95, ε 1e-8), learning rate 5e-6 with cosine decay and no warmup, weight decay 0, gradient clipping at 1.0 and 64 sequences per step. Packed conversations could not attend to each other. Training used the [`marin-community/axolotl`](https://github.com/marin-community/axolotl) fork at `a208d550` with `transformers` 5.12.1, fp32 master weights and bf16 compute, on 16 H100s. Conversations longer than 32,768 tokens were dropped. The router's score-correction bias was not updated. The stage-3 settings are in `axolotl_config.yaml` and `axolotl_run_manifest.json`. | Stage | Data | Steps | |---|---|---:| | 1. Chat | [`nyu-dice-lab/wildchat50m-rewild-sft-385700`](https://huggingface.co/datasets/nyu-dice-lab/wildchat50m-rewild-sft-385700) @ `46a5bb5` | 261 | | 2. Thinking | [`laion/llama-nemotron-science-reasoning-on-canonical-think-full`](https://huggingface.co/datasets/laion/llama-nemotron-science-reasoning-on-canonical-think-full) @ `bae881d` | 628 | | 3. Terminal agent | [`nvidia/Nemotron-Terminal-Corpus`](https://huggingface.co/datasets/nvidia/Nemotron-Terminal-Corpus) @ `a1667c4` | 1,291 (2.71B tokens, of which 1.83B were trained on) | An audit of 2,000 stage-1 conversations confirmed that every assistant turn's closing `<|im_end|>` was a training target, both before and after packing. This export is the bf16 copy of stage 3's final fp32 weights. ## Chat format The chat template is in `chat_template.jinja` and is also embedded in `tokenizer_config.json`. Training and serving used the same template, a ChatML-style template shared with the other models in this experiment. ```text <|im_start|>system {system}<|im_end|> <|im_start|>user {user}<|im_end|> <|im_start|>assistant {reasoning}{answer}<|im_end|> ``` - **Turn end:** `<|im_end|>` (id 11), which is also the tokenizer's end-of-sequence token, so runtimes stop on it by default. `` (2) is a second stop in `generation_config.json`. - **Thinking:** the model decides whether to think; the template doesn't prefill a think block. Reasoning sits between `` (12) and `` (13). These are ordinary vocabulary tokens, not special tokens, so they stay in the returned text by default. - **Multi-turn:** send each earlier reply back as the assistant message's `content`, unchanged, think block included. That renders the same tokens the model was trained on. - **Tool calls** are written as `\n{"name": ..., "arguments": {...}}\n`. Results go back in a `tool` message, rendered as `\n...\n`. - **Tool definitions:** the template does **not** render the `tools` request field, so tools passed that way are silently ignored. Describe the tools and the call format in the system message. A tool-use conversation looks like this: ```python messages = [ {"role": "system", "content": ( "You can call one tool. To call it, write exactly:\n" '\n{"name": "get_weather", "arguments": {"city": ""}}\n\n' "get_weather(city: string) returns the current weather in that city." )}, {"role": "user", "content": "Is it raining in Paris?"}, {"role": "assistant", "content": '\n{"name": "get_weather", "arguments": {"city": "Paris"}}\n'}, {"role": "tool", "content": '{"conditions": "light rain", "temp_c": 11}'}, ] ``` ## Recommended decode `generation_config.json` sets the decode used for every evaluation below; vLLM and `transformers` apply it by default: | Setting | Value | |---|---| | temperature | 0.6 | | top_p | 0.95 | | repetition_penalty | 1.05 | | stop | `<|im_end|>` (11) and `` (2) | | max new tokens | 32,768 for single-turn; 16,384 per agent turn | ## Serving Stock vLLM and `transformers` 5.x support `nemotron_h` natively; no `trust_remote_code` is needed. Evaluation used Marin's vLLM build [`0.0.0.dev20260924+marin.01911be34fac`](https://github.com/marin-community/vllm/releases/tag/marin-vllm-gpu-20260924-01911be34fac), with one model copy per H100 80 GB GPU at a 131,072-token context and up to 32 concurrent sequences. Recent stock vLLM releases that support Nemotron-3-Nano should also work, but they were not tested with this checkpoint. ```sh vllm serve open-athena/Nemotron-3-Nano-30B-A3B-Chat-Aligned-Base-2026.10.30 \ --max-model-len 131072 --max-num-seqs 32 ``` `config.json` sets `mamba_ssm_cache_dtype: float32`. NVIDIA recommends running its Nemotron-3-Nano models in vLLM with `--mamba-ssm-cache-dtype float32`. The evaluation below did not pass that flag. ```python from openai import OpenAI def split_reply(text): """Return (reasoning, answer). reasoning is None without a think block; answer is None if the think block never closed.""" text = text or "" if not text.startswith(""): return None, text reasoning, closed, answer = text[len(""):].partition("") return reasoning, (answer if closed else None) client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused") reply = client.chat.completions.create( model="open-athena/Nemotron-3-Nano-30B-A3B-Chat-Aligned-Base-2026.10.30", messages=[{"role": "user", "content": "How many primes are there below 50?"}], max_tokens=32768, ) reasoning, answer = split_reply(reply.choices[0].message.content) ``` With `transformers`: ```python from transformers import AutoModelForCausalLM, AutoTokenizer name = "open-athena/Nemotron-3-Nano-30B-A3B-Chat-Aligned-Base-2026.10.30" tokenizer = AutoTokenizer.from_pretrained(name) model = AutoModelForCausalLM.from_pretrained(name, dtype="bfloat16", device_map="auto") messages = [{"role": "user", "content": "How many primes are there below 50?"}] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_dict=True, return_tensors="pt" ).to(model.device) output = model.generate(**inputs, max_new_tokens=32768) print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)) ``` Evaluation used no vLLM reasoning or tool-call parser; replies were split as above. `` and `` are single tokens in this vocabulary, which vLLM's `hermes` tool parser requires, but tool calling through that parser was not tested. ## Evaluation and known behavior The SFT had to pass a protocol gate before the model was used in the experiment. `P_chat` is 200 single-turn chat prompts and `P_think` is 200 olympiad-style reasoning prompts, both at the recommended decode. Replies that hit the 32,768-token limit while still inside their think block count as length-limited and are left out of both shares: 1 on `P_chat` and 5 on `P_think`. | Check | Threshold | `P_chat` | `P_think` | |---|---|---:|---:| | Turn ends on `<|im_end|>` | ≥98% | 94.0% | 98.0% | | Think block is balanced: as many `` as ``, ending closed | ≥98% | 40.9% (of 181) | 90.3% (of 195) | | Reply that opens a think block closes it at least once | — | 181 of 182 | 195 of 200 | | Stray special tokens or `` stops | — | 0 | 0 | The model failed this part of the gate. The shortfall was accepted rather than fixed with extra training, since the training-target audit above showed the data was correct. The agentic, tool-call and expert-load checks were run afterwards on this checkpoint. `P_agent` is 32 SWE-bench Verified tasks in the [Terminus-2](https://github.com/laude-institute/terminal-bench) harness, where each agent turn must be a JSON object with `analysis`, `plan` and `commands`. It was served at 131,072 tokens with the recommended decode. | Check | Threshold | Result | |---|---|---| | `P_agent` tasks ending validly (see below) | ≥29 of 32 | 32 of 32: 10 completed, 19 hit the time limit, 3 grader failures | | Agent turns whose JSON parses | ≥95% | **91.1%** (2,450 of 2,689), below threshold | | Tasks solved (not gated) | — | 2 of 29 scored | | Per MoE layer, share of routed experts getting under 10% of the mean load: increase over the base | ≤ +2 points | **+5.47 points** in MoE layer 3 (7 of 128 experts, against none in the base), below threshold; MoE layer 9 is at +2.34 | A task counts as valid if it completes, hits the harness's 3,000-second agent time limit, or ends with the grader failing; time-outs are expected when small models act as agents, and grader failures are not the model's. Any other harness error counts against the model. The routing check used the same 2.16M tokens of mixed text for both models. These shortfalls were also accepted. What this means in use: - **Chat replies often contain extra `` tags.** The model almost always closes its think block and writes an answer. But on 107 of 182 chat replies (and 17 of 200 reasoning replies) it then writes another `` instead of ending the turn, and repeats or revises the answer, sometimes as a JSON object. The balanced-think-block check above fails because of these extra tags. `split_reply` above splits at the first ``, so `answer` holds the first answer followed by the repeats. Cut it at the next `` to keep only the first answer. - **Some replies start with `` and no ``** (18 of 200 chat replies). `split_reply` treats them as answers that begin with a stray tag; strip it. - **About 1 chat reply in 17 doesn't end its turn.** Always set a `max_tokens` limit. - **Agent runs loop.** Half of its agent turns (1,338 of 2,689) repeat the previous reply word for word, and 9 of its 19 time-outs are such loops, often after the model reports that the terminal shows nothing. Watch for repeated turns and cap trajectory length. - **Some routed experts are starved.** In one MoE layer, 7 of 128 experts get under 10% of the mean load, against none in the base. The fine-tune kept the router's score-correction bias frozen, which likely contributes. Expert-routing analyses should take this into account. - Only the checks above were run. There is no broad capability evaluation. ## License The weights are governed by two licenses: - **The base model:** this is a derivative of NVIDIA's Nemotron-3-Nano 30B-A3B base, licensed under the NVIDIA Nemotron Open Model License. A copy is in [LICENSE.NVIDIA-Nemotron-Open-Model-License](LICENSE.NVIDIA-Nemotron-Open-Model-License), and its attribution notice is in [NOTICE](NOTICE). Using or redistributing these weights must comply with that license. - **This repository's modifications:** released under OpenMDW 1.1; the full text is in [LICENSE](LICENSE). The training datasets carry their own licenses; see each dataset's card. Licensed by NVIDIA Corporation under the NVIDIA Nemotron Model License. We release our tools to the public with the intent that they are used safely, ethically, and responsibly. They are meant to contribute to the public's understanding of science, and aid in its advancement. By using this model, you agree to comply with all applicable law.

Full Model Card (README)