Quanta-X-3B: A Reasoning-Focused Small Language Model
Quanta-X-3B, developed by szili2011, is a 3 billion parameter model built on the Qwen 2.5 base architecture. It distinguishes itself by integrating the Phoenix Framework, which combines DoRA (Weight-Decomposed Low-Rank Adaptation) and an aggressive SimPO (Simulated Preference Optimization Beta 2.0) alignment. This unique training approach is designed to instill "System 2" thinking, enabling deeper reasoning capabilities in a lightweight model.
Key Capabilities & Features
- Ouroboros Logic Loop: The model employs a hidden reasoning layer where it internally
plans, drafts, and critiques its thoughts before generating a final response. This process aims to enhance logical consistency and reduce errors. - Diamond-Tier Data Filtering: Quanta-X was trained on a highly curated dataset, including DeepSeek-R1 Traces for logic, OpenR1 Math for verified proofs, Glaive Code V2 for production-ready code, and SlimOrca RP for nuanced storytelling.
- Hyper-Stability: Through SimPO with a Beta of 2.0, the model was heavily penalized for hallucinations or superficial thinking during training, resulting in a preference for admitting ignorance over providing incorrect information.
- Extended Context: Supports a 32k context length, utilizing RoPE scaling.
- Chat Template: Uses the standard Qwen 2.5 ChatML template.
When to Use This Model
- Complex Reasoning Tasks: Ideal for applications requiring more than instant, superficial responses, where logical consistency and internal validation are crucial.
- Problem Solving: Suitable for scenarios where the model needs to "think through" a problem, such as logic puzzles or mathematical challenges.
- Reduced Hallucinations: Beneficial for use cases where factual accuracy and a reluctance to "lie" are paramount, due to its SimPO-driven hyper-stability.
- Resource-Constrained Environments: Offers advanced reasoning capabilities within a 3 billion parameter footprint, making it efficient for deployment where larger models might be impractical.