caiyuchen/DAPO-step-7

Model Overview

The caiyuchen/DAPO-step-7 model is an 8 billion parameter language model built upon the Qwen3-8B-Base architecture. It represents a specific training checkpoint from the research detailed in the paper "On Predictability of Reinforcement Learning Dynamics for Large Language Models" by Yuchen Cai et al. The primary purpose of this model is to facilitate research into the dynamics of reinforcement learning (RL) applied to large language models (LLMs), particularly in the context of improving reasoning capabilities.

Key Research Insights

The underlying research identifies two crucial properties of RL-induced parameter updates:

• Rank-1 Dominance: The most significant improvements in reasoning are captured by the top singular subspace of the parameter update matrix.
• Rank-1 Linear Dynamics: This dominant subspace evolves linearly throughout training, enabling accurate predictions from early checkpoints.

These insights led to the development of AlphaRL, an acceleration framework that extrapolates final parameter updates from a short initial training period. AlphaRL can achieve up to a 2.5x speedup while retaining over 96% of the original reasoning performance.

Use Cases

This model checkpoint is specifically provided for:

• Evaluating RL Dynamics: Researchers can use this model to study and understand how LLM parameters change during RL training.
• Predicting Parameter Evolution: It serves as a tool for investigating the predictability of these parameter dynamics, especially in mathematical reasoning tasks.

Prompt Format

For inference, questions should be formatted with an instruction to reason step-by-step and enclose the final answer in a boxed format, then wrapped using the standard chat template for Qwen3 models.