Tu522004/RD-9B-Distill
RD-9B-Distill is a 9-billion parameter causal language model developed by Tu522004, fine-tuned from Qwen3.5-9B using Reasoning Distillation with Claude Opus as the teacher model. It is optimized for reasoning and problem-solving, demonstrating superior performance over larger 20B models in MMLU-Pro and GPQA Diamond benchmarks while significantly reducing inference token costs. This model is designed for efficient deployment on edge hardware, excelling in complex logical tasks and algorithmic coding.
Loading preview...
RD-9B-Distill: Reasoning-Optimized 9B Model
RD-9B-Distill is a 9-billion parameter causal language model developed by Tu522004, built upon the Qwen3.5-9B base. It leverages Reasoning Distillation with Claude Opus as the teacher model to enhance problem-solving capabilities, specifically targeting the intelligence ceiling of the 9B parameter class for edge hardware accessibility.
Key Innovations & Capabilities
- Shortest Logical Path (Token Optimization): The model learns to find the most efficient reasoning path, reducing average reasoning tokens on MMLU-Pro from 1226 to 434 while improving accuracy.
- Self-Verification Templates: It employs mechanisms to break down complex problems and verify hypotheses, preventing reasoning breakdowns in multi-step logical chains.
- Algorithmic Scaffolding: Provides a strong structural foundation for coding logic, improving performance on algorithmic problems.
Performance Highlights
- MMLU-Pro (Reasoning & Knowledge): Achieved a +30.39% absolute improvement over its base model, scoring 61.11% and outperforming a 20B reference model (47.96%) with significantly lower inference costs.
- GPQA Diamond (PhD-level Expert Reasoning): Demonstrated high stability and accuracy, scoring 62.12% compared to 56.06% for the 20B reference model.
- LiveCodeBench (Coding): Doubled the Pass@1 score of its base model, reaching 34.55%, showcasing robust planning capabilities for algorithmic tasks.
Ideal Use Cases
- Applications requiring strong reasoning and problem-solving on resource-constrained edge devices.
- Tasks benefiting from efficient token usage and reduced inference costs.
- Complex logical tasks, expert-level reasoning, and algorithmic code generation where a smaller, highly optimized model is preferred over larger alternatives.