reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B-Thinking-SFT

Hugging Face
TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:0.8BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Mar 22, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Warm

The reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B-Thinking-SFT is a 0.6 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It was created through a two-stage distillation process: first from a 30B-parameter "Thinking" teacher model to establish a reasoning backbone, then fine-tuned on legal instruction data. This model is optimized for ultra-lightweight reasoning, particularly in legal and STEM domains, and is designed for deployment on resource-constrained devices, with quantized versions under 500MB.

Loading preview...

Overview

This model, reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B-Thinking-SFT, is a 0.6 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It employs a unique two-stage training pipeline. The first stage involves knowledge distillation from a 30B-parameter "Thinking" teacher model (Qwen/Qwen3-30B-A3B-Thinking-2507) using 6,122 STEM chain-of-thought samples. This process focuses on transferring a structured reasoning backbone by emphasizing derivation tokens and the teacher's deliberation landscape. The second stage is supervised fine-tuning on legal instruction data from Alignment-Lab-AI/Lawyer-Instruct, leveraging the learned reasoning structure for legal analysis. This approach allows for a 50x compression ratio compared to the teacher model, making it highly efficient for deployment.

Key Capabilities

  • Ultra-lightweight Reasoning: Designed for efficient reasoning on devices with limited resources, with quantized versions under 500MB.
  • Two-Stage Reasoning Transfer: Benefits from a "Thinking" teacher for deep reasoning structure, followed by domain-specific fine-tuning.
  • Legal and STEM Instruction Following: Excels at tasks requiring structured derivation in STEM and logical argumentation in legal contexts.
  • High Compression: Achieves significant model size reduction while retaining complex reasoning abilities.

Good For

  • Ultra-lightweight reasoning on mobile, edge, and IoT devices.
  • Legal and STEM instruction-following applications.
  • Educational tutoring and embedded inference.
  • Use as a component in multi-model pipelines where reasoning is needed in a small footprint.

Finetunes
2 models