reaperdoesntknow/TopologicalQwen
TopologicalQwen by Convergent Intelligence LLC is a 1.7 billion parameter Qwen3ForCausalLM model with a 40,960 token context length, distilled from Qwen3-30B-A3B using Topological Knowledge Distillation (TKD). This methodology captures structural information beyond standard smooth distillation, enabling the model to perform dual-mental-modality reasoning (explore, examine, respond) for complex problem-solving, particularly in physics CoT tasks.
Loading preview...
TopologicalQwen: Topology-Aware Knowledge Distillation
TopologicalQwen is a 1.7 billion parameter model developed by Convergent Intelligence LLC, distilled from a 30B Qwen3 teacher model. Its core innovation lies in Topological Knowledge Distillation (TKD), a method that goes beyond standard knowledge distillation by decomposing knowledge transfer into three channels: smooth distillation, jump corrections at conceptual boundaries, and drift corrections for residual structure. This allows the model to preserve the teacher's structural understanding, not just its surface statistics.
Key Capabilities & Features
- Topology-Aware Distillation: Utilizes Discrepancy Calculus (DISC) to detect and preserve structural features like topic shifts and reasoning mode transitions, which standard KD methods often blur.
- DualMind Reasoning Format: Trained to respond in a unique
<explore>(derivation),<examine>(self-critique), and<response>(clean answer) format, mimicking a cognitive loop for enhanced problem-solving. - Optimized for Reasoning: Specifically trained on physics CoT datasets (Differential Equations, Theoretical Mechanics, Electromagnetism, General Relativity) to excel in complex reasoning tasks.
- High Context Length: Features a substantial 40,960 token context window.
- Efficient Performance: Achieves advanced reasoning capabilities at a compact 1.7B parameter count, demonstrating the effectiveness of TKD even with premium compute resources (H100, BF16).
What Makes it Different
Unlike other distillation methods that treat the teacher's output as a smooth function, TKD explicitly accounts for discontinuities and structural shifts in the knowledge manifold. This mathematical foundation, detailed in the "Structure Over Scale" methodology, enables TopologicalQwen to produce responses with structural reasoning quality that standard distillation at the same parameter count cannot achieve. It represents a significant advancement in transferring complex reasoning abilities from large teachers to smaller student models.