reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT
reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT is a 2 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It was created through a two-stage process: knowledge distillation from a 30B MoE teacher for STEM reasoning, followed by supervised fine-tuning on legal instruction data. This model excels at structured legal reasoning and STEM problem-solving with step-by-step derivations, designed for efficient application of learned reasoning patterns.
Loading preview...
Model Overview
This model, developed by Convergent Intelligence LLC, is a 2 billion parameter Qwen3-based causal language model. It employs a unique two-stage training pipeline to imbue it with structured reasoning capabilities, first in STEM domains and then applied to legal reasoning.
Key Capabilities
- Structured Reasoning: Achieved through knowledge distillation from a 30B MoE teacher on 6,122 STEM chain-of-thought samples, emphasizing rigorous derivation steps.
- Domain Adaptation: Supervised fine-tuning on legal instruction data leverages the established reasoning backbone to perform legal analysis, case analysis, and statutory interpretation.
- Efficient Deployment: As a dense 1.7B parameter model, it avoids the routing overhead of its 30B MoE teacher while retaining significant knowledge.
- Dual Prompt Formats: Supports both a "Problem: Proof:" format for STEM derivations and an "### Instruction: ### Response:" format for general instruction following and legal tasks.
Good For
- Structured legal reasoning and analysis.
- STEM problem-solving requiring step-by-step derivations.
- Instruction-following in technical domains.
- Educational tutoring and proof drafting.
- Edge and mobile deployment via GGUF quantizations.
- Serving as a component in multi-model pipelines and retrieval-augmented generation (RAG) workflows requiring lightweight reasoning.