Model Overview

SasanoHideo/qwen3-4b-dpo-qwen-cot-merged-rev.01 is a 4 billion parameter language model derived from the Qwen/Qwen3-4B-Instruct-2507 base model. It has undergone Direct Preference Optimization (DPO) using the Unsloth library, specifically targeting improved response alignment and quality.

Key Capabilities

• Enhanced Reasoning: Optimized to improve Chain-of-Thought reasoning, leading to more logical and structured outputs.
• Structured Response Quality: Fine-tuned to align responses with preferred outputs, enhancing the overall quality and coherence of generated text.
• Full Merged Weights: This repository provides the full 16-bit merged weights, eliminating the need for adapter loading and simplifying deployment with transformers.

Training Details

The model was trained for 1 epoch with a learning rate of 1e-07 and a beta value of 0.1. It utilized a maximum sequence length of 1024 during the DPO process. The training data used for DPO was sourced from [u-10bei/dpo-dataset-qwen-cot].

Good For

• Applications requiring improved reasoning and structured output generation.
• Scenarios where response alignment to specific preferences is crucial.
• Developers looking for a readily deployable, merged model without additional adapter configurations.