reaperdoesntknow/Qwen3-1.7B-Coder-Distilled-SFT
reaperdoesntknow/Qwen3-1.7B-Coder-Distilled-SFT is a 1.7 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It was created through a two-stage knowledge distillation process, first from a 30B Coder teacher for a STEM reasoning backbone, then fine-tuned on 54,600 logical inference problems. This model excels at formal reasoning, logical inference, and structured STEM derivation, making it suitable for tasks requiring precise sequential logic and compositional reasoning.
Loading preview...
Model Overview
reaperdoesntknow/Qwen3-1.7B-Coder-Distilled-SFT is a 1.7 billion parameter model from the Qwen3 family, developed by Convergent Intelligence LLC. Its unique training pipeline involves a two-stage distillation process designed to imbue it with strong reasoning capabilities.
Key Capabilities
- Structured Reasoning Backbone: Stage 1 involved knowledge distillation from a 30B Qwen3-Coder teacher, transferring its precise sequential logic, explicit state tracking, and compositional decomposition patterns, particularly for STEM derivations.
- Logical Inference: Stage 2 fine-tuned the model on ~54,600 instruction-response pairs covering propositional logic, logical entailment, and formal inference, activating the latent logical structure from the Coder teacher.
- Proof-Weighted Distillation: Utilizes a novel proof-weighted cross-entropy loss (55%) combined with KL divergence (45%) at T=2.0, amplifying loss on reasoning-critical tokens to foster structural understanding.
- Context Length: Supports a context length of 1024 tokens during training.
Good For
- Logical inference and propositional logic
- Formal reasoning and structured argumentation
- STEM derivation and educational tutoring
- Component in verification pipelines
- Edge deployment via GGUF quantized versions