chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step20

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 7, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step20 is a 1.5 billion parameter Qwen2.5-Instruct model fine-tuned using Proximal Policy Optimization (PPO) for the ALFWorld environment. This specific checkpoint represents optimizer step 20 of the training process, demonstrating early-stage performance with a 13.3% in-training validation success rate. It is designed for research into reinforcement learning applications for language models in interactive text-based environments.

Loading preview...

Model Overview

This model, chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step20, is a 1.5 billion parameter Qwen2.5-Instruct backbone that has undergone Reinforcement Learning (RL) fine-tuning using Proximal Policy Optimization (PPO). The training specifically targets the ALFWorld environment, a text-based interactive world designed to test an agent's ability to follow instructions and interact with objects.

Key Characteristics

  • Backbone: Utilizes Qwen/Qwen2.5-1.5B-Instruct (1.54B parameters, bf16 precision).
  • Training Method: Fine-tuned with PPO, leveraging the langfengQ/verl-agent framework.
  • Checkpoint Specifics: This release corresponds to optimizer step 20 of the training, showing an in-training validation success rate of 13.3% on a 32-task batch.
  • Environment: Optimized for tasks within the ALFWorld alfworld/AlfredTWEnv environment.
  • Training Details: Training was conducted on 2x NVIDIA A100 80GB GPUs, with specific hyperparameters for RL (e.g., GAE for advantage estimation, KL loss with coefficient 0.01) and optimization (actor learning rate 1e-6, critic learning rate 1e-5).

Intended Use Cases

  • RL Research: Ideal for researchers exploring PPO-based fine-tuning of large language models for interactive environments.
  • ALFWorld Benchmarking: Can be used as a baseline or comparison point for agents operating within the ALFWorld environment.
  • Continued Training: The provided training_state/ allows for resuming RL training from this exact optimizer step, enabling further experimentation and performance improvement.

Finetunes
1 models