KS150/dpo-qwen-cot-merged_2
KS150/dpo-qwen-cot-merged_2 is a fine-tuned Qwen3-4B-Instruct-2507 model, optimized using Direct Preference Optimization (DPO) via Unsloth. This 4 billion parameter model focuses on improving reasoning through Chain-of-Thought and enhancing structured response quality. It is designed for applications requiring aligned, high-quality conversational outputs, particularly in reasoning tasks.
Loading preview...
Model Overview
KS150/dpo-qwen-cot-merged_2 is a fine-tuned language model based on Qwen/Qwen3-4B-Instruct-2507. It leverages Direct Preference Optimization (DPO), implemented with the Unsloth library, to enhance its performance.
Key Capabilities
- Improved Reasoning: Optimized to enhance Chain-of-Thought (CoT) reasoning abilities.
- Structured Response Quality: Fine-tuned to produce higher quality and more structured outputs based on preferred responses.
- Direct Use: This model provides full-merged 16-bit weights, allowing direct use with the
transformerslibrary without requiring adapter loading.
Training Details
The model was trained for 5 epochs with a learning rate of 7e-04 and a beta value of 0.1, using a maximum sequence length of 1024. The training data utilized was [u-10bei/dpo-dataset-qwen-cot].
Good For
- Applications requiring enhanced reasoning capabilities.
- Scenarios where high-quality, structured conversational outputs are critical.
- Developers looking for a readily deployable Qwen3-4B variant with DPO-aligned responses.