chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step70
chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step70 is a 1.54 billion parameter Qwen2.5-Instruct model fine-tuned using Proximal Policy Optimization (PPO) for the ALFWorld environment. This specific checkpoint, at optimizer step 70, demonstrates improved success rates in interactive text-based tasks compared to its base model. It is optimized for agent-based reasoning and action generation within simulated environments.
Loading preview...
Model Overview
This model, chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step70, is a Qwen2.5-1.5B-Instruct backbone that has undergone Reinforcement Learning (RL) fine-tuning using Proximal Policy Optimization (PPO). The training was conducted on the ALFWorld environment, a text-based interactive simulation, leveraging the langfengQ/verl-agent framework.
Key Capabilities
- Enhanced Agent Performance: Achieves a standalone evaluation success rate of 35.42% (± 1.61) on 128 ALFWorld
valid_seentasks, significantly outperforming the baseQwen2.5-1.5B-Instructmodel's 2.86% (± 0.37). - Optimized Action Generation: Demonstrates a mean episode length of 39.3 steps, an improvement over the base model's 49.1, indicating more efficient task completion.
- PPO Fine-tuning: Represents a specific checkpoint at optimizer step 70 from a PPO training run, with in-training validation success rates tracked across various steps.
Good For
- Interactive Agent Development: Ideal for research and development of AI agents in text-based interactive environments like ALFWorld.
- RL-based Language Model Applications: Suitable for exploring the application of PPO to fine-tune LLMs for sequential decision-making tasks.
- Benchmarking: Provides a specific checkpoint for comparing RL fine-tuning strategies and their impact on environmental success rates.