n-mitsuyasu/dpo-qwen-cot-merged
n-mitsuyasu/dpo-qwen-cot-merged is a 4 billion parameter language model, fine-tuned from Qwen/Qwen3-4B-Instruct-2507 using Direct Preference Optimization (DPO) via Unsloth. This model is specifically optimized to enhance reasoning capabilities through Chain-of-Thought (CoT) and improve the quality of structured responses. It features a 40960 token context length and is designed for tasks requiring aligned, high-quality reasoning outputs.
Loading preview...
Model Overview
n-mitsuyasu/dpo-qwen-cot-merged is a 4 billion parameter language model derived from Qwen/Qwen3-4B-Instruct-2507. It has been fine-tuned using Direct Preference Optimization (DPO) with the Unsloth library, focusing on aligning its responses with preferred outputs. The model incorporates a 40960 token context length, making it suitable for processing extensive inputs.
Key Capabilities
- Enhanced Reasoning: Optimized to improve Chain-of-Thought (CoT) reasoning, enabling more structured and logical problem-solving.
- Improved Response Quality: DPO training specifically targets the generation of higher quality and more aligned structured responses.
- Full-Merged Weights: The repository provides full-merged 16-bit weights, eliminating the need for adapter loading and simplifying deployment.
Training Details
The model underwent 3 epochs of DPO training with a learning rate of 1e-05 and a beta value of 0.2. It utilized a maximum sequence length of 768 during training. The LoRA configuration (r=8, alpha=16) was merged into the base model. The training data used was u-10bei/dpo-dataset-qwen-cot.
Good For
- Applications requiring robust reasoning and structured output generation.
- Tasks where response alignment and quality are critical.
- Developers seeking a readily deployable model without complex adapter setups.