reaperdoesntknow/Qwen3-1.7B-Coder-Distilled-SFT
reaperdoesntknow/Qwen3-1.7B-Coder-Distilled-SFT is a 1.7 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It was created through a two-stage distillation process: first from a 30B Coder teacher for structured reasoning, then fine-tuned on 54,600 logical inference problems. This model excels at formal logical inference, propositional logic, and STEM derivation, leveraging a 1024-token context length.
Loading preview...
Model Overview
reaperdoesntknow/Qwen3-1.7B-Coder-Distilled-SFT is a 1.7 billion parameter model from Convergent Intelligence LLC, designed for advanced reasoning tasks. Its unique training involved a two-stage process: initial knowledge distillation from a 30B Qwen3-Coder teacher model to establish a strong STEM reasoning backbone, followed by supervised fine-tuning on a dataset of approximately 54,600 logical inference problems. This methodology aims to activate and make explicit the sequential logic, state tracking, and compositional reasoning capabilities inherited from the Coder teacher.
Key Capabilities
- Structured Reasoning: Inherits precise sequential logic and compositional decomposition patterns from a coding-specialized teacher model.
- Logical Inference: Fine-tuned specifically for propositional logic, logical entailment, and formal inference tasks.
- STEM Derivation: Capable of rigorous derivations in various scientific and mathematical domains, including Physics, Linear Algebra, and Advanced Calculus.
- Proof-Weighted Distillation: Utilizes a novel "proof-weighted cross-entropy" loss function during distillation to emphasize reasoning-critical tokens.
Training Methodology
The model's training is grounded in Discrepancy Calculus, a measure-theoretic framework. Stage 1 involved distillation from Qwen3-Coder-30B-A3B-Instruct using 6,122 STEM chain-of-thought samples. Stage 2 applied SFT on the KonstantinDob/logic_inference_dataset for formal logical inference.
Intended Uses
- Logical Inference: Ideal for tasks involving propositional logic and formal reasoning.
- STEM Derivation: Useful for generating rigorous derivations in scientific and engineering problems.
- Educational Tutoring: Can serve as a component for structured argumentation and problem-solving assistance.
- Verification Pipelines: Suitable for integration into systems requiring logical consistency checks.
- Edge Deployment: Quantized GGUF versions are available for efficient local execution.
Limitations
As a 1.7B model, it can generate fluent but incorrect logic and is not intended for general code generation or formal proof verification (e.g., Lean/Coq). Its performance on complex multi-step inferences with many quantifiers may be limited, and users should always verify outputs. The context length is 1024 tokens.