wenlongzhao/ARPO-reproduce_Qwen3-8B_SFT_code-config
The wenlongzhao/ARPO-reproduce_Qwen3-8B_SFT_code-config is an 8 billion parameter Qwen3 model, fine-tuned using Supervised Fine-Tuning (SFT) with the ARPO codebase's LLaMA-Factory configuration. This specific variant is optimized for code-related tasks, featuring a maximum sequence length of 15000 and no weight decay during training. It serves as a research checkpoint for agentic SFT experiments, particularly within the context of the Agentic Reinforced Policy Optimization (ARPO) framework.
Loading preview...
Model Overview
This model, wenlongzhao/ARPO-reproduce_Qwen3-8B_SFT_code-config, is a full-parameter Supervised Fine-Tuning (SFT) of the Qwen3-8B base model. It was developed following the LLaMA-Factory YAML configuration specified in the ARPO codebase. This particular release is the code variant, distinguished by its training parameters tailored for code-related contexts.
Key Characteristics
- Base Model: Qwen/Qwen3-8B, an 8 billion parameter language model.
- Training Data: Utilizes the
dongguanting/ARPO-SFT-54Kdataset, which is the official ARPO SFT mix comprising Tool-Star 54K and STILL data. - Training Procedure: Implemented using LLaMA-Factory for full SFT, leveraging DeepSpeed ZeRO-3 and BF16 precision.
- Code Optimization: Configured with a maximum sequence length of 15000 tokens and zero weight decay, making it suitable for longer code sequences.
- Research Focus: Primarily intended as a research checkpoint for exploring ARPO-style agentic SFT, rather than a standalone chat model.
Training Details
The model was trained for 3 epochs with a learning rate of 7e-6 (cosine schedule, 0.1 warmup ratio) and a global batch size of 16. The final training loss recorded was 0.4241.
Intended Use
This model is designed for research purposes related to agentic SFT within the ARPO framework. Users should adhere to the licenses of both the base Qwen3 model and the ARPO-SFT-54K dataset.