laion/rloo-ab-arm1-tis-seqnorm-step45-8B

TEXT GENERATIONConcurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jun 10, 2026Architecture:Transformer Featherless Exclusive Cold

laion/rloo-ab-arm1-tis-seqnorm-step45-8B is an 8 billion parameter language model developed by laion, fine-tuned using the RLOO-n algorithm with a Qwen3-8B SFT base. This model specifically tests a length-bias-corrected reduction method (seq_mean_token_sum_norm_global) and incorporates TIS (True Importance Sampling). It is designed for research into reinforcement learning from human feedback (RLHF) algorithms, particularly focusing on ablation studies for loss reduction techniques.

Loading preview...

Model Overview

laion/rloo-ab-arm1-tis-seqnorm-step45-8B is an 8 billion parameter model developed by laion, serving as an ablation study within the Reinforcement Learning from Human Feedback (RLHF) framework. It is built upon a laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink (a Qwen3-8B SFT) base model.

Key Characteristics

  • RL Algorithm: Utilizes RLOO-n (advantage_estimator=rloo_n) with 8 samples per prompt.
  • Loss Reduction: Employs seq_mean_token_sum_norm_global, a sequence-mean / seqnorm method designed for length-bias correction.
  • True Importance Sampling (TIS): Activated (use_tis=true) with an importance ratio cap of 2.0.
  • Training Data: Fine-tuned on the DCAgent/exp_rpt_pymethods2test-large dataset.
  • Checkpoint: Represents global_step 45, selected based on the trailing-5 EMA of reward/avg_raw_reward.

Use Cases

This model is primarily intended for researchers and developers interested in:

  • RLHF Ablation Studies: Specifically, investigating the impact of different loss reduction techniques and the role of True Importance Sampling.
  • Understanding Length Bias: Exploring methods to correct length bias in RLHF training.
  • Algorithm Development: Analyzing the performance of RLOO-n and related RL algorithms.

Finetunes
1 models