AlexChen1997/fedagent-webshop-grpo-hardness-std1-qwen2.5-1.5b
AlexChen1997/fedagent-webshop-grpo-hardness-std1-qwen2.5-1.5b is a 1.5 billion parameter Qwen2.5-based language model fine-tuned as a WebShop shopping agent. Developed using federated GRPO (Generalized Reinforcement Learning with Policy Optimization) on the FedAgent verl-0.8 stack, this model is specifically optimized for navigating and making purchases in the WebShop environment. It achieves a 70.3% success rate on WebShop validation episodes, significantly outperforming its base model and a companion PPO twin.
Loading preview...
Model Overview
This model, fedagent-webshop-grpo-hardness-std1-qwen2.5-1.5b, is a specialized WebShop shopping agent built upon the Qwen/Qwen2.5-1.5B-Instruct base model. It was trained using a federated GRPO (Generalized Reinforcement Learning with Policy Optimization) approach over 70 rounds, with 100 clients and a hardness-partitioned WebShop goal pool. The model's primary function is to act as an agent within the WebShop environment, demonstrating strong performance in automated online shopping tasks.
Key Capabilities
- High WebShop Success Rate: Achieves a 70.3% success rate on 64 fixed WebShop validation episodes (at round 68, the best-validation aggregate), a substantial improvement over the base model's 4.7%.
- Federated Learning: Trained using a federated learning setup with 100 clients, ensuring robustness across diverse task distributions.
- GRPO Algorithm: Leverages the GRPO algorithm, outperforming a PPO twin in matched rounds.
- Specialized Agent: Designed specifically for WebShop-style observation/action prompting, making it highly effective for its intended domain.
Good For
- WebShop Automation: Ideal for research and development in automated online shopping agents within the WebShop environment.
- Federated Learning Research: Provides a concrete example of a model trained with federated GRPO, useful for studying federated learning applications in reinforcement learning.
- Benchmarking: Can serve as a strong baseline for comparing new agent architectures or training methodologies for WebShop tasks.