Nomushin/dpo-qwen-cot-merged: DPO-Optimized Qwen3 for Reasoning

This model is a 4 billion parameter variant of the Qwen3 architecture, specifically Qwen/Qwen3-4B-Instruct-2507, that has undergone Direct Preference Optimization (DPO). The fine-tuning process, conducted using the Unsloth library, aimed to align the model's responses with preferred outputs, particularly focusing on enhancing its reasoning (Chain-of-Thought) and the quality of structured responses.

Key Capabilities & Features

• Enhanced Reasoning: Optimized to produce more coherent and logical Chain-of-Thought reasoning.
• Structured Response Quality: Improved ability to generate well-formed and structured outputs based on preference data.
• DPO Fine-tuning: Utilizes Direct Preference Optimization for better alignment with human preferences.
• Full-Merged Weights: The repository provides the full 16-bit merged weights, eliminating the need for adapter loading during deployment.
• Base Model: Built upon the robust Qwen/Qwen3-4B-Instruct-2507 foundation.

Good For

• Applications requiring improved logical reasoning and step-by-step problem-solving.
• Scenarios where generating high-quality, structured text outputs is critical.
• Tasks benefiting from a model aligned with preferred response styles through DPO.