reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B

Hugging Face
TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:0.8BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Mar 22, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Warm

reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B is a 0.6 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC: Research Division. It is distilled from a 30B-parameter "Thinking" teacher model, specifically optimized for generating structured STEM derivations and internal reasoning. This model achieves a 50x parameter compression while retaining deep deliberation structures through a unique distillation process that amplifies loss on reasoning steps, making it suitable for lightweight STEM reasoning on edge devices.

Loading preview...

Overview

This model, reaperdoesntknow/Qwen3-0.6B-Distilled-30B-A3B, is a 0.6 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC: Research Division. It is a highly compressed student model (50x parameter compression) distilled from a 30B-parameter Qwen3-A3B-Thinking teacher. Unlike standard distillation, this model learns from a "Thinking" teacher that generates extended internal reasoning, transferring deeper deliberation structures into the smaller student. It is specifically designed to produce structured STEM derivations, making it ideal for tasks requiring step-by-step reasoning.

Key Capabilities

  • Structured STEM Derivations: Excels at generating detailed, step-by-step proofs and reasoning for STEM problems.
  • Efficient Reasoning: Achieves complex reasoning capabilities in a compact 0.6B parameter size, suitable for resource-constrained environments.
  • "Thinking" Teacher Distillation: Learns from a teacher model that considers multiple reasoning paths, leading to a richer understanding of derivation strategies.
  • Proof-Weighted Loss: Training prioritizes accuracy in reasoning steps over answer formatting, ensuring parameters are allocated to core reasoning ability.

Good For

  • Lightweight STEM Reasoning: Ideal for applications on edge or mobile devices where computational resources are limited.
  • Educational Tutoring: Can assist in generating explanations and derivations for STEM subjects.
  • Proof Drafting: Useful for quickly drafting mathematical or scientific proofs.
  • Multi-Model Pipelines: Can serve as a fast, small reasoner component in larger AI systems.
  • IoT and Embedded Inference: Its small footprint (under 500MB quantized) makes it suitable for embedded systems.

Finetunes
3 models