reaperdoesntknow/TopologicalQwen
TopologicalQwen by Convergent Intelligence LLC is a 1.7 billion parameter Qwen3-based causal language model with a 40,960-token context length, distilled from Qwen3-30B-A3B using Topology-Aware Knowledge Distillation (TKD). Unlike standard methods, TKD captures structural information like topic shifts and reasoning mode transitions, enabling the model to exhibit dual-mental-modality reasoning (explore → examine → respond). It is optimized for complex reasoning tasks, particularly in physics and mathematics, by preserving the teacher's structural understanding.
Loading preview...
Overview
TopologicalQwen is a 1.7 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It is distinguished by its use of Topology-Aware Knowledge Distillation (TKD), a novel methodology that goes beyond standard knowledge distillation by capturing the full structural complexity of the teacher model's output. While most distillation methods only account for smooth variations, TKD also incorporates explicit corrections for discontinuities (jumps) and gradual distributional drift (Cantor/singular-continuous components), ensuring a more faithful transfer of complex reasoning capabilities from a larger teacher model (Qwen3-30B-A3B).
Key Capabilities
- Topology-Aware Knowledge Distillation (TKD): Preserves structural reasoning quality by detecting and accounting for conceptual boundaries and distributional shifts in the teacher's output, which standard KD methods miss.
- Dual-Mental-Modality Reasoning: Trained in a unique "DualMind" format, the model generates responses through a cognitive loop of derivation (
<explore>), self-critique (<examine>), and synthesis (<response>), mimicking advanced problem-solving. - Complex Reasoning: Excels in tasks requiring structured thought, particularly in physics and mathematics, due to its specialized training on CoT (Chain-of-Thought) datasets in these domains.
- Efficient Performance: Achieves high structural reasoning quality at a compact 1.7 billion parameters, demonstrating that advanced methodology can yield superior results even with premium compute.
Good For
- Applications requiring structured, multi-step reasoning and problem-solving, especially in scientific or technical fields.
- Use cases where interpretability of the reasoning process is valuable, thanks to its explicit
<explore>,<examine>, and<response>output format. - Developers seeking a highly efficient model that retains complex cognitive abilities typically found in much larger models, leveraging advanced distillation techniques.
- Research into knowledge distillation methodologies and the transfer of complex cognitive structures in LLMs.