curtburk/nemotron3-nano-kd-distilled

TEXT GENERATIONPricing:Input $0.2 / Output $0.8Concurrent Unit Cost:2Model Size:30BQuant:FP8Context Size:32kPublished:Aug 28, 2026License:otherArchitecture:Transformer Featherless Exclusive Cold

The curtburk/nemotron3-nano-kd-distilled model is a 30 billion parameter Nemotron 3 Nano architecture, fine-tuned by curtburk using LoRA with reasoning traces distilled from DeepSeek-V4-Flash. This model significantly improves coding problem termination and accuracy, achieving 63.4% pass@1 on held-out coding problems, which is 92.5% of the teacher's score at 9.5x fewer parameters. It is specifically optimized for code generation and problem-solving, particularly in ensuring complete code output.

Loading preview...

Overview

This model, curtburk/nemotron3-nano-kd-distilled, is a 30 billion parameter version of NVIDIA's Nemotron 3 Nano, fine-tuned by curtburk. It leverages LoRA with reasoning traces distilled from the much larger DeepSeek-V4-Flash (284B) model. The primary goal of this distillation was to improve the base model's ability to complete coding problems, specifically addressing issues where the base model would analyze problems correctly but fail to produce complete code.

Key Capabilities & Improvements

  • Enhanced Code Completion: The model significantly improves the termination rate of responses, from 34.3% in the base model to 93.0%, ensuring more complete code outputs.
  • High Pass@1 on Coding: Achieves a pass@1 score of 63.4% on held-out coding problems (including APPS and HumanEval+), a substantial increase from the base model's 23.0%. This represents 92.5% of the teacher model's performance with significantly fewer parameters.
  • Efficiency: While improving completion, the mean completion tokens are reduced from 9,736 (base) to 3,125, making it more efficient than the base model.
  • Hardware Efficiency: The entire corpus construction, training, merging, and evaluation were performed on a single NVIDIA GB300 Grace Blackwell Ultra GPU.

What Makes This Model Different?

Unlike many models that focus on algorithmic improvements, this model's key differentiator is its ability to reliably finish coding tasks. It learned the decision to stop analyzing and commit to an answer, which is particularly beneficial for hard problems where the base model often spiraled without resolution. On easy problems, it matches the teacher's performance exactly.

Limitations

  • Single Run: Results are based on a single training run and checkpoint.
  • Teacher Contamination: The teacher model generated traces for the evaluation problems, meaning the teacher's score is a contaminated ceiling.
  • Format Adherence: The model did not learn to reproduce the specific markdown structure used in the training corpus (Analysis / Approach / Implementation Plan / Solution / Verification).
  • Scope: Not evaluated for general capability, chat quality, or long-context behavior; these aspects may have regressed.

Finetunes
1 models