Phase-Technologies/qwen2.5-3b-claude-distilled-reasoning-dpo

Overview

This model, qwen2.5-3b-claude-distilled-reasoning-dpo, is a 3.09 billion parameter causal language model built upon Qwen/Qwen2.5-3B-Instruct. It undergoes a two-stage post-training alignment process to enhance its reasoning capabilities and factual accuracy.

Key Capabilities

• Distilled Chain-of-Thought (CoT): Excels at step-by-step reasoning for mathematical equations, coding challenges, and logic puzzles, derived from Claude 3.5 Sonnet monologue traces.
• Factually Grounded: Demonstrates high performance on graduate/research-level physics and mathematics questions, significantly reducing hallucinations common in basic SFT models.
• Alignment: Utilizes Direct Preference Optimization (DPO) with argilla/ultrafeedback-binarized-preferences-cleaned to suppress scientific hallucinations and token repetition loops.
• Low Memory Footprint: Designed to run efficiently in FP16/SDPA, requiring approximately 6GB VRAM on a single consumer GPU.
• ChatML Ready: Fully compatible with standard Qwen2.5 ChatML chat templates and system prompt instructions.

Good For

• Applications requiring robust, step-by-step reasoning in mathematics, logic, and code synthesis.
• Tasks where factual accuracy in scientific and technical domains is critical.
• Deployment on resource-constrained hardware, such as single consumer GPUs, due to its optimized memory usage.