reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT

Hugging Face
TEXT GENERATIONConcurrency Cost:1Model Size:2BQuant:BF16Ctx Length:32kPublished:Mar 22, 2026License:apache-2.0Architecture:Transformer Open Weights Warm

reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT is a 1.7 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It was created through a two-stage process: knowledge distillation from a 30B MoE teacher for STEM chain-of-thought reasoning, followed by supervised fine-tuning on legal instruction data. This model excels at structured legal reasoning and STEM problem-solving with step-by-step derivations, making it suitable for technical instruction-following and educational applications.

Loading preview...

Model Overview

This model, reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT, is a 1.7 billion parameter Qwen3-based language model developed by Convergent Intelligence LLC. Its unique development involved a two-stage training pipeline designed to impart structured reasoning capabilities and domain-specific knowledge.

Key Capabilities

  • Two-Stage Training: Initially distilled from a 30B MoE teacher (Qwen3-30B-A3B-Instruct-2507) using 6,122 STEM chain-of-thought samples. This stage focused on building a robust reasoning backbone by prioritizing learning derivation steps over memorizing answers, utilizing a proof-weighted cross-entropy loss and KL divergence. The second stage involved supervised fine-tuning on legal instruction data (Alignment-Lab-AI/Lawyer-Instruct) to apply the learned reasoning patterns to a new domain.
  • Structured Reasoning: Designed to produce structured, step-by-step derivations in STEM problems and logical argumentation in legal contexts.
  • Dual Prompt Formats: Supports both a STEM derivation format (for rigorous problem-solving) and an instruction-following format (for general questions and legal reasoning).
  • Lightweight Deployment: Available in GGUF quantized versions for efficient local and edge deployment.

Good For

  • Structured legal reasoning and analysis.
  • STEM problem-solving requiring detailed, step-by-step derivations.
  • Instruction-following across technical domains.
  • Educational tutoring and proof drafting.
  • Edge/mobile deployment via GGUF and integration into multi-model pipelines requiring lightweight reasoning.

Finetunes
2 models