wenlongzhao/ARPO-reproduce_Qwen3-8B_SFT_paper-config
wenlongzhao/ARPO-reproduce_Qwen3-8B_SFT_paper-config is an 8 billion parameter instruction-tuned model based on Qwen/Qwen3-8B, specifically fine-tuned following the cold-start recipe from the Agentic Reinforced Policy Optimization (ARPO) paper. This variant uses a maximum sequence length of 4096 and a weight decay of 0.1, trained on the ARPO-SFT-54K dataset. It serves as a research checkpoint for ARPO-style agentic Supervised Fine-Tuning (SFT) experiments.
Loading preview...
Overview
This model, wenlongzhao/ARPO-reproduce_Qwen3-8B_SFT_paper-config, is an 8 billion parameter Supervised Fine-Tuning (SFT) of the Qwen3-8B base model. It was developed by wenlongzhao as a reproduction of the "cold-start recipe" detailed in Appendix E.2 of the Agentic Reinforced Policy Optimization (ARPO) paper. This specific variant aligns with the paper's configuration, featuring a maximum sequence length of 4096 and a weight decay of 0.1.
Training Details
The model was trained using LLaMA-Factory with full SFT, DeepSpeed ZeRO-3, BF16 precision, and FlashAttention-2. The training utilized the dongguanting/ARPO-SFT-54K dataset, which is the official ARPO SFT mix comprising Tool-Star 54K and STILL data. Key hyperparameters included a learning rate of 7e-6, a cosine LR schedule with a 0.1 warmup ratio, 3 epochs, and a global batch size of 128. The final training loss achieved was 0.5775.
Intended Use
This model is primarily intended as a research checkpoint for experiments related to ARPO-style agentic SFT. It has not been evaluated as a standalone chat model. Users should adhere to the licenses of both the base Qwen3-8B model and the ARPO-SFT-54K dataset.