sonodd/qwen3-4b-structeval-sft-v4-lr2e5-merged
The sonodd/qwen3-4b-structeval-sft-v4-lr2e5-merged model is a 4 billion parameter language model based on the Qwen3-4B-Instruct-2507 architecture. It integrates a Supervised Fine-Tuning (SFT) LoRA adapter, specifically sonodd/qwen3-4b-structeval-sft-v4-lr2e5, into the base model. This merged full model is primarily designed for use as a base in a SFT to DPO (Direct Preference Optimization) pipeline, facilitating further preference alignment training. It offers a 32768 token context length, making it suitable for tasks requiring extensive context understanding.
Loading preview...
Model Overview
The sonodd/qwen3-4b-structeval-sft-v4-lr2e5-merged is a 4 billion parameter model built upon the Qwen/Qwen3-4B-Instruct-2507 base architecture. This model is a full merge of a Supervised Fine-Tuning (SFT) LoRA adapter, sonodd/qwen3-4b-structeval-sft-v4-lr2e5, into its base, performed using the merge_and_unload() method in float16 precision.
Key Capabilities
- Integrated SFT: Incorporates a pre-trained SFT adapter, providing a strong foundation for subsequent training stages.
- DPO Pipeline Readiness: Specifically configured to serve as the
DPO_BASE_MODELin a SFT to DPO training pipeline, streamlining the process of preference alignment. - Qwen3 Architecture: Benefits from the robust capabilities of the Qwen3-4B-Instruct-2507 base model.
Good For
- Direct Preference Optimization (DPO) Training: Ideal for developers looking to perform DPO on a model that has already undergone supervised fine-tuning.
- Custom Model Development: Provides a merged checkpoint that can be further fine-tuned or adapted for specific downstream tasks requiring a strong SFT foundation.
- Research and Experimentation: Useful for exploring the effects of SFT and DPO in combination, particularly within the Qwen3 ecosystem.