chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step80

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 7, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step80 is a 1.54 billion parameter Qwen2.5-Instruct model fine-tuned using Proximal Policy Optimization (PPO) for improved performance in the ALFWorld environment. This model, specifically checkpoint step 80, demonstrates enhanced success rates in text-based interactive tasks compared to its base model. It is optimized for agent-based reasoning and action generation within simulated environments like ALFWorld.

Loading preview...

Model Overview

This model, chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step80, is a Qwen2.5-1.5B-Instruct backbone that has undergone Reinforcement Learning (RL) fine-tuning using Proximal Policy Optimization (PPO). The training specifically targeted the ALFWorld environment, aiming to improve the model's ability to navigate and complete tasks in text-based interactive simulations.

Key Capabilities & Performance

  • Enhanced ALFWorld Performance: At optimizer step 80, the model achieves a 36.20% ± 1.33 success rate on 128 valid_seen ALFWorld tasks, a significant improvement over the base Qwen2.5-1.5B-Instruct model's 2.86% ± 0.37.
  • Efficient Action Generation: It demonstrates a lower mean episode length (38.3) compared to the base model (49.1), indicating more direct task completion.
  • PPO Fine-tuning: Utilizes PPO with specific hyperparameters, including a critic learning rate of 1e-5 and an invalid action penalty, to optimize policy learning.
  • Context Handling: Supports max_prompt_length of 2048 and max_response_length of 512 tokens.

Ideal Use Cases

  • Agent Development: Suitable for researchers and developers working on AI agents for interactive text-based environments.
  • Reinforcement Learning Research: Provides a checkpoint for further experimentation with PPO-based fine-tuning on language models.
  • ALFWorld Benchmarking: Can be used as a baseline or for comparative studies within the ALFWorld environment.

Finetunes
1 models