reaperdoesntknow/TopologicalQwen
TopologicalQwen by Convergent Intelligence LLC is a 1.7 billion parameter Qwen3-based causal language model with a 40,960 token context length. It utilizes Topological Knowledge Distillation (TKD) to capture structural knowledge from a 30B teacher model, going beyond standard KL divergence by incorporating jump and drift corrections. This model is specifically designed for complex reasoning tasks, learning to generate and self-critique proofs in a 'DualMind' format, making it suitable for applications requiring robust, structured analytical output.
Loading preview...
TopologicalQwen: Topology-Aware Knowledge Distillation
TopologicalQwen is a 1.7 billion parameter model developed by Convergent Intelligence LLC, distilled from a Qwen3-30B-A3B teacher using a novel Topological Knowledge Distillation (TKD) methodology. Unlike traditional KD methods that only capture smooth variations, TKD decomposes knowledge transfer into three channels: smooth distillation, explicit jump corrections at conceptual boundaries, and drift corrections for subtle distributional shifts. This allows the model to preserve the teacher's structural understanding, not just its surface statistics.
Key Capabilities & Features
- Topology-Aware Learning: Employs Discrepancy Calculus (DISC) to detect and preserve structural features in the teacher's output, such as topic shifts and reasoning mode transitions.
- DualMind Reasoning Format: Trained to generate responses in a structured
<explore>(derivation),<examine>(self-critique), and<response>(clean answer) format, mimicking a cognitive loop. - Physics CoT Training: Fine-tuned on a specialized dataset of physics Chain-of-Thought examples, including differential equations, theoretical mechanics, electromagnetism, and general relativity.
- Advanced Distillation Pipeline: Features a 4-phase TKD pipeline with topology-guided adaptive windowing and curriculum-ordered continuous KD, including proof-weighted loss.
- Efficient Performance: Achieves structural reasoning quality comparable to much larger models at a significantly smaller 1.7B parameter count, demonstrating the efficacy of the TKD methodology.
Ideal Use Cases
- Complex Problem Solving: Excels in tasks requiring structured, multi-step reasoning and self-correction, particularly in scientific or mathematical domains.
- Proof Generation & Verification: Suitable for generating and critiquing proofs or derivations, leveraging its DualMind format.
- Knowledge Transfer Research: Demonstrates a novel approach to knowledge distillation, making it valuable for researchers exploring advanced model compression techniques.