laion/ablation-pymethods2test-seqmean-arm0-15-8B
The laion/ablation-pymethods2test-seqmean-arm0-15-8B is an 8 billion parameter causal language model, derived from a Qwen3-8B SFT base model, with a 32768 token context length. Developed by laion, this model is a specific checkpoint from an RL (SkyRL GRPO) ablation study, focusing on a sequence-mean loss reduction with an RLOO-n advantage estimator. It is designed for research into reinforcement learning optimization methods for large language models, particularly exploring different policy loss configurations.
Loading preview...
Model Overview
The laion/ablation-pymethods2test-seqmean-arm0-15-8B is an 8 billion parameter language model, representing a specific checkpoint from an ablation study conducted by laion. It is built upon a Qwen3-8B SFT base model and utilizes a 32768 token context length.
Key Characteristics
- Reinforcement Learning (RL) Focus: This model is a product of an RL (SkyRL GRPO) ablation, specifically from the
sequence-mean / RLOO-n (arm0)study. - Policy Loss Configuration: It distinguishes itself by using
loss_reduction=sequence_meanwith theadvantage_estimator=rloo_n(RLOO-n) estimator for its policy loss, contrasting with the token-mean reduction found in other series. - Training Details: Trained using SkyRL GRPO on the DCAgent/exp_rpt_pymethods2test-large dataset, this is the
global_step_15checkpoint. - Reproducibility: The
rl_config.jsonwithin the repository provides the exact launch configuration for reproducibility.
Intended Use Cases
This model is primarily intended for research and development in the field of reinforcement learning for large language models. It is particularly useful for:
- Investigating the impact of different policy loss reduction methods (e.g., sequence-mean vs. token-mean).
- Studying the performance of the RLOO-n advantage estimator.
- Reproducing and extending the ablation studies conducted by laion on RL optimization techniques.