reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B

Hugging Face
TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:0.8BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Mar 22, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Warm

reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B is a 0.6 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC: Research Division. This model is distilled from a 30B-parameter 'Thinking' teacher model, specifically optimized for STEM chain-of-thought reasoning. It features a unique proof-weighted loss function that amplifies penalties for errors in reasoning steps, making it highly effective for generating structured STEM derivations and suitable for lightweight reasoning on edge devices.

Loading preview...

Overview

This model, reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B, is a 0.6 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC: Research Division. It represents a 50x parameter compression from its 30B-parameter teacher, Qwen/Qwen3-30B-A3B-Thinking-2507. The core innovation lies in its distillation process, which leverages a "Thinking" teacher variant that generates extended internal reasoning, transferring deeper deliberation structures to the smaller student model. This results in a model under 500MB (quantized) capable of producing structured STEM derivations.

Key Capabilities

  • STEM Chain-of-Thought Reasoning: Distilled from a teacher model that generates rich, extended reasoning traces, enabling the student to learn deliberation, not just answers.
  • Proof-Weighted Loss: Derivation tokens (Proof: to Final Answer:) receive amplified loss (2.5x decaying to 1.5x), ensuring parameters are allocated to reasoning capability rather than just answer formatting.
  • Lightweight & Efficient: At 0.6B parameters, it's designed for efficient inference, making it suitable for edge and mobile deployments.
  • Structured Derivations: Excels at generating step-by-step mathematical and scientific proofs and derivations.

Good For

  • Lightweight STEM Reasoning: Ideal for applications requiring mathematical or scientific problem-solving on resource-constrained devices.
  • Educational Tutoring: Can assist in generating explanations and derivations for STEM problems.
  • Proof Drafting: Useful for initial drafting of proofs or complex derivations.
  • Multi-Model Pipelines: Functions effectively as a small, fast reasoner component within larger AI systems.
  • IoT and Embedded Inference: Its small footprint allows for deployment in embedded systems and IoT devices.

Finetunes
3 models