wenlongzhao/ARPO-reproduce_Qwen2.5-3B-Instruct_SFT_paper-config

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:3.1BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Sep 4, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

wenlongzhao/ARPO-reproduce_Qwen2.5-3B-Instruct_SFT_paper-config is a 3.1 billion parameter instruction-tuned model based on Qwen2.5-3B-Instruct, developed by wenlongzhao. It is fine-tuned using the cold-start recipe from the Agentic Reinforced Policy Optimization (ARPO) paper, specifically configured with a maximum sequence length of 4096 and a weight decay of 0.1. This model is intended as a research checkpoint for ARPO-style agentic Supervised Fine-Tuning (SFT) experiments.

Loading preview...

Overview

This model, wenlongzhao/ARPO-reproduce_Qwen2.5-3B-Instruct_SFT_paper-config, is a full-parameter Supervised Fine-Tuning (SFT) of the Qwen/Qwen2.5-3B-Instruct base model. It implements the 'paper' variant of the cold-start recipe detailed in Appendix E.2 of the Agentic Reinforced Policy Optimization (ARPO) research paper. A companion 'code' variant is also available.

Training Details

  • Base Model: Qwen2.5-3B-Instruct
  • Training Data: dongguanting/ARPO-SFT-54K, which combines Tool-Star 54K and STILL datasets.
  • Framework: Trained using LLaMA-Factory with full SFT, DeepSpeed ZeRO-3, BF16 precision, and FlashAttention-2.
  • Key Hyperparameters:
    • Learning rate: 7e-6 (cosine schedule, 0.1 warmup ratio)
    • Epochs: 3
    • Global batch size: 128
    • Max sequence length: 4096
    • Weight decay: 0.1
    • Final train loss: 0.6852

Intended Use

This model serves primarily as a research checkpoint for experiments related to ARPO-style agentic SFT. It has not been evaluated for performance as a standalone chat model. Users should adhere to the licenses of both the base model and the SFT dataset.

Finetunes
1 models