reaperdoesntknow/TopologicalQwen
TopologicalQwen by Convergent Intelligence LLC is a 1.7 billion parameter Qwen3-based causal language model with a 40,960-token context length, distilled from Qwen3-30B-A3B using Topology-Aware Knowledge Distillation (TKD). Unlike standard methods, TKD captures structural information by decomposing knowledge transfer into smooth, jump, and drift components, enabling the model to preserve the teacher's reasoning structure. It is specifically designed for complex reasoning tasks, particularly in physics, and utilizes a DualMind format for structured exploration, examination, and response generation.
Loading preview...
TopologicalQwen: Topology-Aware Knowledge Distillation for Enhanced Reasoning
TopologicalQwen is a 1.7 billion parameter Qwen3-based model developed by Convergent Intelligence LLC, distinguished by its use of Topological Knowledge Distillation (TKD). This advanced distillation methodology, grounded in Discrepancy Calculus, allows the model to capture and preserve the structural nuances of a larger 30B-parameter teacher model (Qwen3-30B-A3B) that traditional knowledge distillation methods miss. While standard KD only accounts for continuous variations in the teacher's output, TKD explicitly corrects for discontinuities (jumps) and gradual drifts in the knowledge manifold, leading to superior structural reasoning capabilities at a significantly smaller scale.
Key Capabilities
- Topology-Aware Knowledge Transfer: Utilizes a novel approach to distill knowledge, preserving complex reasoning structures by identifying and correcting for 'jumps' and 'drifts' in the teacher's distribution.
- DualMind Reasoning Format: Trained to generate responses in a structured
<explore>(derivation),<examine>(self-critique), and<response>(clean answer) format, mimicking a cognitive loop for robust problem-solving. - Physics CoT Specialization: Fine-tuned on physics-related Chain-of-Thought (CoT) datasets, including differential equations, theoretical mechanics, electromagnetism, and general relativity.
- Efficient Structural Preservation: Achieves high-quality structural reasoning at 1.7 billion parameters, demonstrating that advanced methodology can yield results comparable to much larger models.
Good For
- Complex Reasoning Tasks: Ideal for applications requiring deep, structured reasoning, particularly in scientific and mathematical domains.
- Structured Output Generation: Suitable for use cases where a clear, multi-stage thought process (exploration, critique, synthesis) is beneficial or required.
- Resource-Constrained Environments: Offers advanced reasoning capabilities in a compact 1.7B parameter model, making it efficient for deployment where larger models are impractical.