chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step60

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 7, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step60 model is a 1.54 billion parameter Qwen2.5-Instruct backbone fine-tuned using Proximal Policy Optimization (PPO) for agentic tasks within the ALFWorld environment. This specific checkpoint, at optimizer step 60, demonstrates a 30.99% success rate on ALFWorld's valid_seen tasks, significantly outperforming the base model's 2.86%. It is optimized for sequential decision-making and action generation in text-based interactive environments.

Loading preview...

Overview

This model, chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step60, is a specialized variant of the Qwen/Qwen2.5-1.5B-Instruct language model, fine-tuned using Proximal Policy Optimization (PPO) for improved performance in the ALFWorld environment. It leverages a 1.54 billion parameter Qwen2.5 backbone and has been trained for 60 optimizer steps, demonstrating a substantial increase in task success rate compared to its base model.

Key Capabilities

  • Enhanced Agentic Performance: Achieves a 30.99% success rate (± 2.42) on 128 ALFWorld valid_seen tasks, a significant improvement over the base Qwen2.5-1.5B-Instruct model's 2.86%.
  • Efficient Action Generation: Exhibits a high format rate of 98.5% for generated actions, ensuring parseable outputs for environmental interaction.
  • Reinforcement Learning Fine-tuning: Utilizes PPO with specific hyperparameters, including GAE for advantage estimation and a low-variance KL loss, to optimize policy learning.
  • Context Handling: Supports a max_prompt_length of 2048 tokens and max_response_length of 512 tokens, suitable for complex interactive scenarios.

Training Details

This checkpoint represents optimizer step 60 of a PPO training run, which involved restarting from previous checkpoints for operational reasons. The training curve shows a continuous improvement in validation success rate, reaching 35.2% at this step. The model was trained on 2x NVIDIA A100 80GB GPUs, with a critic learning rate of 1e-5 and an actor learning rate of 1e-6.

Good for

  • Research in RLHF and Agentic LLMs: Ideal for researchers exploring reinforcement learning from human feedback (RLHF) or developing LLM-based agents for interactive environments.
  • ALFWorld Task Solving: Specifically designed and evaluated for tasks within the ALFWorld environment, making it a strong candidate for similar text-based game or simulation tasks.
  • Comparative Studies: Useful for comparing PPO-tuned models against other fine-tuning methods or base models in agentic settings.

Finetunes
1 models