reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B

Hugging Face
TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:0.8BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Mar 22, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Warm

reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B is a 0.6 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. This model is distilled from a 30B-parameter 'Thinking' teacher model, specifically optimized for STEM chain-of-thought reasoning. It achieves a 50x parameter compression while retaining structured derivation capabilities, making it suitable for lightweight STEM reasoning tasks on resource-constrained devices. The unique distillation process emphasizes learning deliberation and reasoning steps over just final answers.

Loading preview...

Overview

This model, reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B, is a 0.6 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It represents a 50x parameter compression, distilled from a 30B-parameter Qwen3-A3B-Thinking teacher model. The core innovation lies in its distillation methodology, which focuses on transferring deep deliberation structure for STEM chain-of-thought reasoning, rather than just final answers.

Key Differentiators

  • Thinking Teacher Distillation: Unlike standard distillation from 'Instruct' teachers, this model learns from a 'Thinking' variant of Qwen3-30B-A3B. This teacher generates extended internal reasoning, providing a richer landscape of derivation strategies for the student model to learn from, emphasizing the 'how' over just the 'what'.
  • Proof-Weighted Loss: During training, tokens within the derivation region (Proof: to Final Answer:) receive amplified loss (2.5x decaying to 1.5x). This ensures that the model's limited parameters are primarily allocated to reasoning capability rather than just answer formatting.
  • STEM CoT Focus: Trained on 6,122 STEM chain-of-thought samples across 12 domains, it is specifically designed to produce structured derivations for scientific, technical, engineering, and mathematical problems.

Intended Uses

  • Lightweight STEM reasoning on edge/mobile devices
  • Educational tutoring and proof drafting
  • Component in multi-model pipelines requiring a small, fast reasoner
  • IoT and embedded inference applications

Limitations

Due to its 0.6B parameter size, the model may struggle with multi-step proofs requiring more than ~8 reasoning steps, complex multi-variable problems, or domains underrepresented in its training data. Verification of outputs is always recommended.

Finetunes
3 models