Model Overview

KS150/dpo-qwen-cot-merged_2 is a fine-tuned language model based on Qwen/Qwen3-4B-Instruct-2507. It leverages Direct Preference Optimization (DPO), implemented with the Unsloth library, to enhance its performance.

Key Capabilities

• Improved Reasoning: Optimized to enhance Chain-of-Thought (CoT) reasoning abilities.
• Structured Response Quality: Fine-tuned to produce higher quality and more structured outputs based on preferred responses.
• Direct Use: This model provides full-merged 16-bit weights, allowing direct use with the transformers library without requiring adapter loading.

Training Details

The model was trained for 5 epochs with a learning rate of 7e-04 and a beta value of 0.1, using a maximum sequence length of 1024. The training data utilized was [u-10bei/dpo-dataset-qwen-cot].

Good For

• Applications requiring enhanced reasoning capabilities.
• Scenarios where high-quality, structured conversational outputs are critical.
• Developers looking for a readily deployable Qwen3-4B variant with DPO-aligned responses.