reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT

Hugging Face
TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Mar 22, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Warm

reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT is a 1.7 billion parameter Qwen3-based model developed by Convergent Intelligence LLC. It was created through a two-stage process: knowledge distillation from a 30B MoE teacher on STEM chain-of-thought data to build a reasoning backbone, followed by supervised fine-tuning on legal instruction data. This unique training approach prioritizes learning structured reasoning before applying it to specific domains, making it adept at both STEM problem-solving with derivations and structured legal analysis.

Loading preview...

Overview

reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B-SFT is a 1.7 billion parameter language model from Convergent Intelligence LLC, built on the Qwen3 architecture. Its development involved a unique two-stage training pipeline designed to impart robust reasoning capabilities and domain-specific knowledge.

Key Capabilities

  • Structured Reasoning: The model first underwent knowledge distillation from a 30B Mixture-of-Experts (MoE) teacher using 6,122 STEM chain-of-thought samples. This stage focused on teaching the model how to reason by prioritizing derivation steps over final answers, using a proof-weighted cross-entropy loss and KL divergence to capture the teacher's full probability landscape.
  • Domain Adaptation: Following distillation, the model was fine-tuned on legal instruction data (Alignment-Lab-AI/Lawyer-Instruct). This stage layered legal domain knowledge and instruction-following capabilities onto the established reasoning backbone, leveraging the structural similarities between mathematical and legal reasoning.
  • Dual Prompt Formats: It supports two distinct prompt formats: one for STEM derivations (e.g., "Problem: ... Proof:") and another for general instruction-following and legal queries (e.g., "### Instruction: ... ### Response:").
  • Lightweight Deployment: As a 1.7B parameter model, it is suitable for edge and mobile deployments, with GGUF quantized versions available.

Good for

  • Structured legal reasoning and analysis.
  • STEM problem-solving requiring step-by-step derivations.
  • Instruction-following in technical domains.
  • Educational tutoring and proof drafting.
  • Integration into multi-model pipelines or retrieval-augmented generation (RAG) workflows where lightweight reasoning is beneficial.

Finetunes
2 models