reaperdoesntknow/TopologicalQwen
TopologicalQwen by Convergent Intelligence LLC is a 1.7 billion parameter Qwen3ForCausalLM model with a 40,960 token context length. It is distilled from Qwen3-30B-A3B using Topological Knowledge Distillation (TKD), a novel methodology that preserves structural information by accounting for smooth, jump, and drift components of the teacher's output distribution. This model excels at complex reasoning tasks, particularly in physics, by generating responses in a unique DualMind format of exploration, self-critique, and synthesis.
Loading preview...
TopologicalQwen: Topology-Aware Knowledge Distillation
TopologicalQwen is a 1.7 billion parameter model developed by Convergent Intelligence LLC, distilled from a 30B Qwen3 teacher model. Its core innovation lies in Topological Knowledge Distillation (TKD), a method that goes beyond standard distillation by decomposing knowledge transfer into three channels: smooth distillation, jump corrections at conceptual boundaries, and drift corrections for subtle distributional shifts. This approach ensures the student model retains the teacher's structural understanding, not just surface statistics.
Key Capabilities & Features
- Topological Knowledge Distillation (TKD): Utilizes Discrepancy Calculus (DISC) to detect and preserve structural features in the teacher's knowledge, such as topic shifts and reasoning mode transitions.
- DualMind Format: Trained to generate responses in a unique
<explore>(derivation),<examine>(verification/self-critique), and<response>(clean answer) cognitive loop, enhancing reasoning quality. - Architecture: Based on Qwen3ForCausalLM with 28 layers, 16 attention heads (GQA), and a substantial 40,960 token context length.
- Specialized Training: Distilled from Qwen3-30B-A3B-Thinking-2507 using a 4-phase TKD pipeline with physics CoT datasets (Differential Equations, Theoretical Mechanics, Electromagnetism, General Relativity).
What Makes This Model Different
Unlike conventional knowledge distillation methods that treat the teacher's output as a smooth function, TKD explicitly accounts for discontinuities and subtle drifts in the knowledge manifold. This allows TopologicalQwen, despite its small size, to exhibit a dual-mental-modality reasoning capability and structural quality that standard distillation cannot achieve. The methodology, grounded in mathematical theory, demonstrates that preserving topological structure can yield superior reasoning performance even with limited parameters, especially when provided with premium compute resources like H100 GPUs.
Good For
- Complex reasoning and problem-solving, particularly in scientific and mathematical domains.
- Applications requiring structured thought processes and self-correction.
- Research into advanced knowledge distillation techniques and cognitive architectures in LLMs.