ryzdfm/qwen2.5-coder-3b-claude_opus_4.6-distilled

QwOpus-4.6-Coder-3B: Claude Opus 4.6 Reasoning Distilled

This model, ryzdfm/qwen2.5-coder-3b-claude_opus_4.6-distilled, is a 3 billion parameter coding model built upon the robust Qwen2.5-Coder-3B-Instruct base. It has been fine-tuned using Supervised Fine-Tuning (SFT) with LoRA, distilling the structured, step-by-step reasoning capabilities of Claude 4.6 Opus.

Key Capabilities

• Structured Reasoning: Emulates Claude Opus's thought process, using <think> tags to break down problems before providing a final answer.
• Code Generation: Leverages its Qwen2.5-Coder foundation for strong performance in languages like Python and JavaScript, and for algorithmic problem-solving.
• Math & Logic: Capable of solving multi-step mathematical and logical problems with verification.
• Fast Local Inference: Optimized for consumer hardware, achieving ~88 tokens/sec on an RTX 3050 and requiring only ~2.1 GB VRAM for Q4_K_M quantization, making it suitable for devices with 4GB VRAM.

Training Details

The model was trained for 1 epoch on a Tesla T4 GPU using 3,209 high-quality Claude reasoning samples from datasets like nohurry/Opus-4.6-Reasoning-3000x-filtered and TeichAI/claude-4.5-opus-high-reasoning-250x. This process focused on embedding the structured reasoning style into a compact model.

Limitations

Due to its 3B scale and single-epoch training, the model may struggle with highly complex, multi-file code generation or deep reasoning tasks compared to much larger models. Like all LLMs, it carries a risk of hallucination, and outputs should always be verified.