reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT
reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT is a 1.7 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It was created through a two-stage process: first, knowledge distillation from a 30B MoE teacher on 6,122 STEM chain-of-thought samples to build a structured reasoning backbone, followed by supervised fine-tuning on legal instruction data. This model excels at structured legal reasoning and STEM problem-solving with step-by-step derivations, offering instruction-following capabilities within a 1024 token context window.
Loading preview...
Model Overview
This model, developed by Convergent Intelligence LLC, is a 1.7 billion parameter Qwen3-based language model designed for structured reasoning. It was trained using a unique two-stage methodology: initially, knowledge distillation from a 30B MoE teacher model using 6,122 STEM chain-of-thought samples to establish a robust reasoning foundation. Subsequently, it underwent supervised fine-tuning on legal instruction data to integrate domain-specific knowledge and enhance instruction-following capabilities.
Key Capabilities
- Structured Reasoning: Emphasizes rigorous, step-by-step derivations in STEM problems due to proof-weighted distillation.
- Domain Adaptation: Applies learned reasoning patterns to new domains, specifically legal analysis, rather than just memorizing templates.
- Dual Prompt Formats: Supports both STEM derivation prompts and general instruction-following prompts.
- Lightweight Deployment: Available in GGUF format for efficient local and edge deployment.
Good For
- Structured legal reasoning and analysis.
- STEM problem-solving requiring detailed derivations.
- Instruction-following in technical domains.
- Educational tutoring and proof drafting.
- Integration into multi-model pipelines and retrieval-augmented workflows needing lightweight reasoning.