laion/seqnorm-tis-shaped-40-8B
The laion/seqnorm-tis-shaped-40-8B model is an 8 billion parameter RL checkpoint derived from a Qwen3-8B SFT base model, specifically fine-tuned for agentic coding tasks. It utilizes a sequence-normalized RLOO-n objective combined with Truncated Importance Sampling (TIS) and a shaped reward based on test passing rates. This model is optimized for improving code generation and problem-solving within agentic environments, particularly on the `pymethods2test-large` dataset, offering enhanced performance in automated coding scenarios.
Loading preview...
Model Overview
laion/seqnorm-tis-shaped-40-8B is an 8 billion parameter Reinforcement Learning (RL) checkpoint, representing step 40 of 80 in its training progression. It is built upon a Qwen3-8B SFT base model (laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink) and has a context length of 32768 tokens.
Key Capabilities & Training
This model is specifically fine-tuned for agentic coding tasks, leveraging the pymethods2test-large dataset. Its training objective incorporates a unique combination of sequence-normalized RLOO-n and Truncated Importance Sampling (TIS). A key differentiator is its shaped reward function, which directly optimizes for the percentage of tests passing, making it highly effective for code generation and validation. The training was conducted using the SkyRL framework on a Jupiter 56-GPU cluster.
Unique Aspects
This model is the shaped-reward successor to the a3 RLOO-n series, focusing on an ablation study with a combined sequence normalization and TIS objective. Training traces, including the last episode of each trial, are available as a companion dataset: penfever/seqnorm-tis-shaped.
Good For
- Automated code generation and problem-solving in agentic environments.
- Tasks requiring high accuracy in passing test cases for generated code.
- Research and development in RL-based code synthesis and optimization.