Rina1001/dpo-qwen-cot-merged
Rina1001/dpo-qwen-cot-merged is a 4 billion parameter causal language model, fine-tuned from Qwen/Qwen3-4B-Instruct-2507 using Direct Preference Optimization (DPO). This model specializes in improving reasoning capabilities, particularly Chain-of-Thought (CoT), and generating high-quality structured responses. Its primary use case is enhancing conversational AI and complex task completion through better alignment with preferred outputs.
Loading preview...
Overview
Rina1001/dpo-qwen-cot-merged is a 4 billion parameter language model, fine-tuned from the Qwen/Qwen3-4B-Instruct-2507 base model. It leverages Direct Preference Optimization (DPO) via the Unsloth library to enhance its response quality. This model provides full-merged 16-bit weights, eliminating the need for adapter loading.
Key Capabilities
- Improved Reasoning: Optimized specifically to enhance Chain-of-Thought (CoT) reasoning, allowing for more structured and logical outputs.
- High-Quality Responses: Aligned through DPO to produce preferred outputs, leading to better overall response quality.
- Direct Usage: As a fully merged model, it can be used directly with the
transformerslibrary without additional configuration.
Good For
- Applications requiring enhanced reasoning and problem-solving capabilities.
- Scenarios where structured and aligned responses are critical.
- Developers looking for a 4B parameter model with improved conversational and task completion quality through DPO.