reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT

Hugging Face
TEXT GENERATIONConcurrency Cost:1Model Size:2BQuant:BF16Ctx Length:32kTool Calling:SupportedPublished:Mar 22, 2026License:apache-2.0Architecture:Transformer Open Weights Warm

The reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT is a 1.7 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It was created through a two-stage process: first, knowledge distillation from a 30B MoE teacher on 6,122 STEM chain-of-thought samples to build a structured reasoning backbone, followed by supervised fine-tuning on legal instruction data. This model excels at structured legal reasoning and STEM problem-solving with step-by-step derivations, making it suitable for technical instruction-following and educational applications.

Loading preview...

Model Overview

This model, developed by Convergent Intelligence LLC, is a 1.7 billion parameter Qwen3-based language model designed for structured reasoning. It was trained using a novel two-stage approach: initially, knowledge distillation from a 30B-parameter Qwen3 MoE teacher model on 6,122 STEM chain-of-thought samples, followed by supervised fine-tuning on legal instruction data. This methodology aims to first instill a robust reasoning backbone and then layer domain-specific knowledge, transferring structured thinking patterns across different fields.

Key Capabilities

  • Structured Reasoning: Employs a proof-weighted cross-entropy loss during distillation to prioritize learning rigorous derivation steps in STEM problems, rather than just memorizing answers.
  • Domain Transfer: Applies learned reasoning structures from STEM to legal analysis, enabling structured argumentation and premise identification in legal contexts.
  • Dual Prompt Formats: Supports both a STEM derivation format (for step-by-step problem-solving) and an instruction-following format (for general questions and legal reasoning).
  • Lightweight Deployment: At 1.7B parameters, it's suitable for edge/mobile deployment, with GGUF quantized versions available.

Good For

  • Structured legal reasoning and analysis.
  • STEM problem-solving requiring step-by-step derivations.
  • Technical instruction-following across various domains.
  • Educational tutoring and proof drafting.
  • Integration into multi-model pipelines and retrieval-augmented workflows needing lightweight reasoning.

Finetunes
2 models