Model Overview

The takami2022/qwen3-4b-dpo-v1 is a 4 billion parameter language model derived from takami2022/qwen3-4b-sft-merged-v2v5ver1. Its key differentiator is the application of Direct Preference Optimization (DPO) during fine-tuning, utilizing the Unsloth library.

Key Characteristics

• Base Model: Fine-tuned from takami2022/qwen3-4b-sft-merged-v2v5ver1.
• Optimization Method: Employs DPO to enhance alignment with human preferences.
• Weights: Contains full-merged 16-bit weights, simplifying deployment as no separate adapter loading is required.
• Training Configuration: Trained for 1 epoch with a learning rate of 1e-07, a beta value of 0.1, and a maximum sequence length of 1024 tokens. LoRA configuration (r=16, alpha=32) was used and subsequently merged.

Good For

• Applications where outputs aligned with specific human preferences are crucial.
• Scenarios requiring a compact 4B parameter model with DPO-enhanced response quality.
• Developers seeking a ready-to-use model without the complexity of adapter management.