chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step30

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 7, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step30 model is a 1.54 billion parameter language model, fine-tuned from Qwen/Qwen2.5-1.5B-Instruct using Proximal Policy Optimization (PPO). It is specifically optimized for interactive decision-making tasks within the ALFWorld environment, demonstrating a 14.8% validation success rate at optimizer step 30. This model is designed for research and development in reinforcement learning for language agents, particularly for tasks requiring sequential actions and environmental interaction.

Loading preview...

Overview

This model, chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step30, is a 1.54 billion parameter language model based on Qwen/Qwen2.5-1.5B-Instruct. It has undergone Reinforcement Learning (RL) fine-tuning using Proximal Policy Optimization (PPO) on the ALFWorld environment, specifically alfworld/AlfredTWEnv. This particular checkpoint represents the model's state at optimizer step 30.

Key Capabilities & Training Details

  • RL Fine-tuning: Utilizes PPO for optimizing agent behavior in interactive environments.
  • Environment: Specialized for tasks within the ALFWorld text-based game environment.
  • Performance at Step 30: Achieved an in-training validation success rate of 14.8% on a 32-task validation batch.
  • Hyperparameters: Key RL parameters include GAE for advantage estimation, KL loss with a coefficient of 0.01, and an invalid action penalty. The actor learning rate is 1e-6, and the critic learning rate is 1e-5.
  • Hardware: Trained on 2x NVIDIA A100 80GB GPUs.
  • Checkpoint Availability: This is one of several checkpoints available from the same training run, allowing for exploration of different optimization stages.

Intended Use Cases

This model is primarily intended for:

  • Research in RL for LLMs: Studying the effects of PPO fine-tuning on language models for interactive tasks.
  • Agent Development: As a base for developing and evaluating language agents in environments like ALFWorld.
  • Understanding Training Dynamics: Analyzing the training curve and hyperparameter effects on model performance in sequential decision-making.

Finetunes
1 models