modrill/code-think-q4b-20260908

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:4BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Sep 9, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

modrill/code-think-q4b-20260908 is a 4 billion parameter research checkpoint model based on Qwen/Qwen3-4B-Base, fine-tuned using a Mix Distillation approach. It specializes in code generation, particularly for Python, and demonstrates improved pass@1 scores on LiveCodeBench-derived problems compared to its base model. This model is designed for "think" mode inference, enabling enhanced reasoning capabilities for programming tasks.

Loading preview...

Overview

modrill/code-think-q4b-20260908 is a 4 billion parameter research checkpoint model, built upon the Qwen/Qwen3-4B-Base architecture. It has been fine-tuned using a novel Mix Distillation technique, combining traces from both a stronger 30B parameter teacher model and a same-size 4B parameter teacher. This approach addresses the challenge of small models learning effectively from much stronger reasoners, as highlighted in recent research.

Key Capabilities

  • Enhanced Code Generation: Specifically optimized for generating Python programs, demonstrating improved performance on coding benchmarks.
  • "Think" Mode Inference: Supports a specialized "think" mode, activated by enable_thinking=True, which allows the model to engage in a reasoning process before generating code.
  • Improved Reasoning: Achieves a pass@1 score of 73/256 on a DEV256 LiveCodeBench-derived split, outperforming the bare Qwen3-4B-Base (63/256) under similar conditions.
  • Context Length: Utilizes a substantial context window of 32768 tokens, beneficial for complex coding problems.

Good for

  • Python Code Generation: Ideal for developers and researchers focused on generating correct Python programs from problem specifications.
  • Exploring Distillation Techniques: Useful for understanding the impact of mixed-teacher distillation on smaller language models, particularly in reasoning-intensive tasks.
  • Research and Development: Serves as a valuable research checkpoint for further experimentation in code-thinking and reasoning capabilities of LLMs.

This model is provided as a research checkpoint and not a production-ready product, licensed under Apache-2.0.

Finetunes
1 models