takami2022/qwen3-4b-dpo-v1
TEXT GENERATIONConcurrency Cost:1Model Size:4BQuant:BF16Ctx Length:32kPublished:Mar 1, 2026License:apache-2.0Architecture:Transformer Open Weights Warm
The takami2022/qwen3-4b-dpo-v1 is a 4 billion parameter language model, fine-tuned from takami2022/qwen3-4b-sft-merged-v2v5ver1 using Direct Preference Optimization (DPO). This model incorporates full-merged 16-bit weights, eliminating the need for adapter loading. Its DPO fine-tuning aims to align its responses more closely with human preferences, making it suitable for tasks requiring nuanced and preferred outputs.
Loading preview...
Model Overview
The takami2022/qwen3-4b-dpo-v1 is a 4 billion parameter language model derived from takami2022/qwen3-4b-sft-merged-v2v5ver1. Its key differentiator is the application of Direct Preference Optimization (DPO) during fine-tuning, utilizing the Unsloth library.
Key Characteristics
- Base Model: Fine-tuned from
takami2022/qwen3-4b-sft-merged-v2v5ver1. - Optimization Method: Employs DPO to enhance alignment with human preferences.
- Weights: Contains full-merged 16-bit weights, simplifying deployment as no separate adapter loading is required.
- Training Configuration: Trained for 1 epoch with a learning rate of 1e-07, a beta value of 0.1, and a maximum sequence length of 1024 tokens. LoRA configuration (r=16, alpha=32) was used and subsequently merged.
Good For
- Applications where outputs aligned with specific human preferences are crucial.
- Scenarios requiring a compact 4B parameter model with DPO-enhanced response quality.
- Developers seeking a ready-to-use model without the complexity of adapter management.