Model Overview

This model, ryowatanabe240215/qwen3-4b-structured-output-lora_ver10-2_merge_dpo, is a 4 billion parameter variant of the Qwen3 architecture, specifically fine-tuned from Qwen/Qwen3-4B-Instruct-2507. It has been optimized using Direct Preference Optimization (DPO) via the Unsloth library, focusing on aligning its responses with preferred outputs.

Key Capabilities

• Enhanced Reasoning: Optimized to improve Chain-of-Thought reasoning, leading to more logical and coherent outputs.
• Structured Output Quality: Fine-tuned to produce high-quality structured responses, making it suitable for tasks requiring specific formats.
• DPO Alignment: Benefits from DPO training, which aligns the model's behavior with human preferences based on a dedicated preference dataset.
• Full-Merged Weights: Provided as full-merged 16-bit weights, eliminating the need for adapter loading and simplifying deployment.

Good For

• Applications requiring improved reasoning capabilities.
• Use cases where structured and precise outputs are critical.
• Developers looking for a Qwen3-based model with enhanced alignment to preferred response styles.

Training Details

The model underwent 1 epoch of DPO training with a learning rate of 1e-07 and a beta of 0.1, using a maximum sequence length of 1024. The training data utilized was u-10bei/dpo-dataset-qwen-cot.