Overview

Rina1001/dpo-qwen-cot-merged is a 4 billion parameter language model, fine-tuned from the Qwen/Qwen3-4B-Instruct-2507 base model. It leverages Direct Preference Optimization (DPO) via the Unsloth library to enhance its response quality. This model provides full-merged 16-bit weights, eliminating the need for adapter loading.

Key Capabilities

• Improved Reasoning: Optimized specifically to enhance Chain-of-Thought (CoT) reasoning, allowing for more structured and logical outputs.
• High-Quality Responses: Aligned through DPO to produce preferred outputs, leading to better overall response quality.
• Direct Usage: As a fully merged model, it can be used directly with the transformers library without additional configuration.

Good For

• Applications requiring enhanced reasoning and problem-solving capabilities.
• Scenarios where structured and aligned responses are critical.
• Developers looking for a 4B parameter model with improved conversational and task completion quality through DPO.