reaperdoesntknow/Qwen3-1.7B-Coder-Distilled-SFT
reaperdoesntknow/Qwen3-1.7B-Coder-Distilled-SFT is a 1.7 billion parameter Qwen3-based model developed by Convergent Intelligence LLC, specialized in structured reasoning. It was created through a two-stage distillation process: first from a 30B Coder teacher for a STEM reasoning backbone, then fine-tuned on 54,600 logical inference problems. This model excels at formal reasoning, logical inference, and structured STEM derivation, leveraging the precise sequential logic and compositional decomposition learned from its Coder teacher.
Loading preview...
Model Overview
reaperdoesntknow/Qwen3-1.7B-Coder-Distilled-SFT is a 1.7 billion parameter model from Convergent Intelligence LLC, built on the Qwen3 architecture. Its unique development process involves a two-stage knowledge distillation and fine-tuning approach, designed to imbue it with strong structured reasoning capabilities.
Key Capabilities
- Structured Reasoning Backbone: Distilled from a 30B Qwen3-Coder teacher, it inherits precise sequential logic, explicit state tracking, and compositional decomposition patterns crucial for rigorous derivations.
- Logical Inference: Supervised fine-tuning on ~54,600 logical inference problems (propositional logic, logical entailment) activates and refines its formal reasoning abilities.
- STEM Derivation: Optimized for solving and rigorously deriving solutions in STEM fields, leveraging the structured approach learned from the Coder teacher.
- Proof-Weighted Distillation: Utilizes a novel proof-weighted cross-entropy loss (55%) combined with KL divergence (45%) to emphasize reasoning-critical tokens during distillation, ensuring structural understanding.
Training Methodology
The model's training is grounded in Discrepancy Calculus, a measure-theoretic framework. Stage 1 involved knowledge distillation from a Qwen3-Coder-30B-A3B-Instruct teacher using 6,122 STEM chain-of-thought samples. Stage 2 applied supervised fine-tuning on the KonstantinDob/logic_inference_dataset to explicitly activate formal logical inference.
Good For
- Logical inference and propositional logic tasks
- Formal reasoning and structured argumentation
- STEM derivation and educational tutoring
- Component in verification pipelines
- Edge deployment via GGUF (quantized versions available)
Limitations
As a 1.7B model, it can generate fluent but incorrect logic, and its logical inference performance is strongest on patterns seen during training. It is not intended for general code generation or formal proof verification (e.g., Lean/Coq), nor for tasks requiring context beyond its 1024-token limit.