laion/symclip-30-8B
The laion/symclip-30-8B model is an 8 billion parameter language model based on a Qwen3-8B SFT base, fine-tuned using SkyRL GRPO. It utilizes a symmetric PPO clip range (0.2 / 0.2) with sequence-normalized loss reduction and Truncated Importance Sampling (TIS) on untruncated rollouts. This model is specifically developed for reinforcement learning tasks, focusing on policy optimization with a unique symmetric clipping strategy. Its primary application lies in research and development for advanced RL policy training.
Loading preview...
Model Overview
laion/symclip-30-8B is an 8 billion parameter model derived from a Qwen3-8B SFT base. It was developed by laion through reinforcement learning using the SkyRL GRPO algorithm. This model is notable for its specific training configuration, which employs a symmetric PPO clip range (0.2 / 0.2), sequence-normalized loss reduction, and Truncated Importance Sampling (TIS) applied to untruncated rollouts. This approach differentiates it from other configurations that might use asymmetric clipping.
Training Details
The model was trained on the DCAgent/exp_rpt_pymethods2test-large dataset. The checkpoint global_step_30 was selected based on a trailing-5 Exponential Moving Average (EMA) of the average raw reward. Training was conducted on 56 GPUs (14x GH200 nodes) at JSC Jupiter. The training run was stopped at step 78, with the reward peaking around step 30 before declining. Training traces and detailed metrics, including per-step CSVs and raw logs, are available in companion datasets and within the repository.
Key Characteristics
- Symmetric PPO Clipping: Utilizes a 0.2 / 0.2 clip range for policy loss.
- Sequence-Normalized Loss: Employs
seq_mean_token_sum_norm_globalfor loss reduction. - Truncated Importance Sampling (TIS): Applied to rollout log-probabilities on untruncated rollouts.
- Base Model: Built upon
laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink(a Qwen3-8B SFT).
Intended Use
This model is primarily intended for researchers and developers working on reinforcement learning policy optimization, particularly those interested in exploring the effects of symmetric PPO clipping and Truncated Importance Sampling in RL training. It serves as a specific checkpoint from an ablation series, offering insights into advanced RL training methodologies.