Model Overview

The sonodd/qwen3-4b-structeval-sft-v4-lr2e5-merged is a 4 billion parameter model built upon the Qwen/Qwen3-4B-Instruct-2507 base architecture. This model is a full merge of a Supervised Fine-Tuning (SFT) LoRA adapter, sonodd/qwen3-4b-structeval-sft-v4-lr2e5, into its base, performed using the merge_and_unload() method in float16 precision.

Key Capabilities

• Integrated SFT: Incorporates a pre-trained SFT adapter, providing a strong foundation for subsequent training stages.
• DPO Pipeline Readiness: Specifically configured to serve as the DPO_BASE_MODEL in a SFT to DPO training pipeline, streamlining the process of preference alignment.
• Qwen3 Architecture: Benefits from the robust capabilities of the Qwen3-4B-Instruct-2507 base model.

Good For

• Direct Preference Optimization (DPO) Training: Ideal for developers looking to perform DPO on a model that has already undergone supervised fine-tuning.
• Custom Model Development: Provides a merged checkpoint that can be further fine-tuned or adapted for specific downstream tasks requiring a strong SFT foundation.
• Research and Experimentation: Useful for exploring the effects of SFT and DPO in combination, particularly within the Qwen3 ecosystem.