ryowatanabe240215/qwen3-4b-structured-output-lora_ver10-2_merge_dpo
The ryowatanabe240215/qwen3-4b-structured-output-lora_ver10-2_merge_dpo is a 4 billion parameter Qwen3-based instruction-tuned causal language model, fine-tuned by ryowatanabe240215. It leverages Direct Preference Optimization (DPO) to enhance reasoning (Chain-of-Thought) and structured output quality. This model is specifically optimized for generating preferred, well-structured responses, making it suitable for applications requiring precise and logical outputs.
Loading preview...
Model Overview
This model, ryowatanabe240215/qwen3-4b-structured-output-lora_ver10-2_merge_dpo, is a 4 billion parameter variant of the Qwen3 architecture, specifically fine-tuned from Qwen/Qwen3-4B-Instruct-2507. It has been optimized using Direct Preference Optimization (DPO) via the Unsloth library, focusing on aligning its responses with preferred outputs.
Key Capabilities
- Enhanced Reasoning: Optimized to improve Chain-of-Thought reasoning, leading to more logical and coherent outputs.
- Structured Output Quality: Fine-tuned to produce high-quality structured responses, making it suitable for tasks requiring specific formats.
- DPO Alignment: Benefits from DPO training, which aligns the model's behavior with human preferences based on a dedicated preference dataset.
- Full-Merged Weights: Provided as full-merged 16-bit weights, eliminating the need for adapter loading and simplifying deployment.
Good For
- Applications requiring improved reasoning capabilities.
- Use cases where structured and precise outputs are critical.
- Developers looking for a Qwen3-based model with enhanced alignment to preferred response styles.
Training Details
The model underwent 1 epoch of DPO training with a learning rate of 1e-07 and a beta of 0.1, using a maximum sequence length of 1024. The training data utilized was u-10bei/dpo-dataset-qwen-cot.