Nomushin/dpo-qwen-cot-merged
Nomushin/dpo-qwen-cot-merged is a 4 billion parameter Qwen3-based causal language model fine-tuned using Direct Preference Optimization (DPO) via Unsloth. This model is specifically optimized for improving reasoning capabilities through Chain-of-Thought (CoT) and generating high-quality structured responses. It leverages a 32768 token context length and is designed for applications requiring enhanced logical coherence and aligned outputs.
Loading preview...
Nomushin/dpo-qwen-cot-merged: DPO-Optimized Qwen3 for Reasoning
This model is a 4 billion parameter variant of the Qwen3 architecture, specifically Qwen/Qwen3-4B-Instruct-2507, that has undergone Direct Preference Optimization (DPO). The fine-tuning process, conducted using the Unsloth library, aimed to align the model's responses with preferred outputs, particularly focusing on enhancing its reasoning (Chain-of-Thought) and the quality of structured responses.
Key Capabilities & Features
- Enhanced Reasoning: Optimized to produce more coherent and logical Chain-of-Thought reasoning.
- Structured Response Quality: Improved ability to generate well-formed and structured outputs based on preference data.
- DPO Fine-tuning: Utilizes Direct Preference Optimization for better alignment with human preferences.
- Full-Merged Weights: The repository provides the full 16-bit merged weights, eliminating the need for adapter loading during deployment.
- Base Model: Built upon the robust
Qwen/Qwen3-4B-Instruct-2507foundation.
Good For
- Applications requiring improved logical reasoning and step-by-step problem-solving.
- Scenarios where generating high-quality, structured text outputs is critical.
- Tasks benefiting from a model aligned with preferred response styles through DPO.