laion/ablation-pymethods2test-seqnorm-15-8B
The laion/ablation-pymethods2test-seqnorm-15-8B is an 8 billion parameter language model, RL-trained by laion using SkyRL on a Qwen3-8B SFT base. It is specifically optimized for the `exp_rpt_pymethods2test-large` task set, focusing on length-bias ablation with a sequence normalization approach. This model is designed for research into reinforcement learning optimization methods for language models.
Loading preview...
Overview
laion/ablation-pymethods2test-seqnorm-15-8B is an 8 billion parameter model developed by laion, focusing on reinforcement learning optimization. It is built upon a laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink base, which is a Qwen3-8B SFT model. The training utilized the SkyRL framework with FSDP2 and vLLM on Jupiter GH200 hardware, specifically employing an RLOO-n advantage estimator and seq_mean_token_sum_norm_global loss reduction.
Key Characteristics
- Base Model: Qwen3-8B SFT variant (
laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink). - Training Objective: RL-trained for length-bias ablation using a sequence normalization (seqnorm) approach.
- Dataset: Optimized on the
DCAgent/exp_rpt_pymethods2test-largetask set. - Training Framework: SkyRL, leveraging advanced RL techniques like RLOO-n advantage estimator.
- Context Length: Supports a context length of 32768 tokens.
Intended Use Cases
This model is primarily intended for:
- Reinforcement Learning Research: Investigating the effects of length-bias ablation and sequence normalization in RL-trained language models.
- Methodology Testing: Evaluating the
exp_rpt_pymethods2test-largetask set with specific RL optimization strategies. - Comparative Analysis: Serving as a baseline or comparison point for other RL optimization techniques in similar domains.