reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B

Hugging Face
TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:0.8BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Mar 22, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Warm

The reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B model is a 0.6 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It is distilled from a 30B-parameter 'Thinking' teacher model, specifically optimized for STEM chain-of-thought reasoning. This model uniquely employs proof-weighted loss during distillation to prioritize learning structured derivations, making it highly effective for lightweight STEM problem-solving and educational applications.

Loading preview...

Overview

This model, reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B, is a 0.6 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It represents a 50x parameter compression from its 30B-parameter Qwen3-A3B-Thinking teacher model. The distillation process uniquely focuses on transferring deep deliberation structures by using a 'Thinking' teacher that generates extended internal reasoning, rather than just final answers. This approach, combined with a proof-weighted loss function, ensures the student model learns to produce structured STEM derivations.

Key Capabilities

  • Specialized STEM Reasoning: Distilled specifically for chain-of-thought (CoT) reasoning in STEM domains, learning deliberation strategies from a larger 'Thinking' teacher.
  • Proof-Weighted Distillation: Utilizes a novel loss function that amplifies penalties for errors within the derivation steps (2.5x decaying to 1.5x), ensuring parameters are allocated to reasoning capability.
  • High Compression: Achieves a 50x parameter reduction from a 30B teacher to a 0.6B student, resulting in a model under 500MB quantized.
  • Structured Output: Designed to produce structured STEM derivations, learning the 'how' of reasoning, not just the 'what'.

Good for

  • Lightweight STEM reasoning on edge/mobile devices.
  • Educational tutoring and proof drafting.
  • Component in multi-model pipelines requiring a small, fast reasoner.
  • IoT and embedded inference applications where resource constraints are critical.

Finetunes
3 models