chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step140
This model, chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step140, is a 1.5 billion parameter Qwen2.5-Instruct backbone model fine-tuned using Proximal Policy Optimization (PPO) for the ALFWorld environment. It specializes in interactive text-based game environments, demonstrating a significant improvement in task success rate compared to its base model. This specific checkpoint represents optimizer step 140, achieving a 62.24% success rate on ALFWorld valid_seen tasks.
Loading preview...
Overview
This model is a 1.5 billion parameter Qwen/Qwen2.5-1.5B-Instruct backbone that has undergone Reinforcement Learning (RL) fine-tuning using Proximal Policy Optimization (PPO) specifically for the ALFWorld environment. Developed by chanyoungkim, this particular checkpoint is from optimizer step 140 of the training process, utilizing the langfengQ/verl-agent framework.
Key Capabilities & Performance
- ALFWorld Task Success: Achieves a 62.24% ± 3.27 success rate on 128 ALFWorld
valid_seentasks, a substantial improvement over the baseQwen2.5-1.5B-Instructmodel's 2.86% ± 0.37. - Efficiency: Demonstrates a mean episode length of 27.6 steps, compared to 49.1 for the base model, indicating more efficient task completion.
- Action Formatting: Maintains a high action format rate of 97.8%.
- Training Method: Fine-tuned with PPO, incorporating GAE for advantage estimation and a KL loss with a coefficient of 0.01.
When to Use This Model
- Interactive Text-Based Environments: Ideal for applications requiring an agent to navigate and interact within text-based game environments like ALFWorld.
- RL Research: Useful for researchers exploring PPO-based fine-tuning of large language models for embodied AI tasks.
- Agent Development: Provides a strong baseline for developing agents that can interpret and act upon natural language instructions in simulated worlds.