reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT

Hugging Face
TEXT GENERATIONConcurrent Unit Cost:1Model Size:2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Mar 22, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Warm

reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT is a 1.7 billion parameter Qwen3-based model developed by Convergent Intelligence LLC. It was created through a two-stage process: knowledge distillation from a 30B MoE teacher for STEM chain-of-thought reasoning, followed by supervised fine-tuning on legal instruction data. This model excels at structured legal reasoning and STEM problem-solving with step-by-step derivations, designed for efficient deployment.

Loading preview...

Model Overview

This model, developed by Convergent Intelligence LLC, is a 1.7 billion parameter Qwen3-based language model engineered for structured reasoning. Its unique two-stage training pipeline first distills knowledge from a 30B Mixture-of-Experts (MoE) teacher model using 6,122 STEM chain-of-thought samples. This initial stage, utilizing a proof-weighted cross-entropy loss and KL divergence, focuses on teaching the model rigorous derivation and reasoning patterns. The second stage involves supervised fine-tuning on legal instruction data, leveraging the established reasoning backbone to apply structured analysis to legal concepts.

Key Capabilities

  • Structured Reasoning: Excels at producing step-by-step derivations in STEM fields (physics, linear algebra, differential equations, etc.).
  • Legal Analysis: Applies learned reasoning structures to legal instruction-following, case analysis, and statutory interpretation.
  • Efficient Deployment: A 1.7B parameter model designed for lightweight applications, with quantized GGUF versions available for local/edge deployment.
  • Dual Prompt Formats: Supports both a "Problem: ... Proof:" format for derivations and an "### Instruction: ... ### Response:" format for general instruction-following.

Good For

  • Structured legal reasoning and analysis.
  • STEM problem-solving requiring detailed, step-by-step derivations.
  • Instruction-following in technical domains.
  • Educational tutoring and proof drafting.
  • Edge/mobile deployment via GGUF quantizations.
  • Component in multi-model pipelines and retrieval-augmented generation (RAG) workflows where lightweight reasoning is crucial.

Finetunes
2 models