chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step10

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 7, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step10 is a 1.54 billion parameter Qwen2.5-Instruct model fine-tuned using Proximal Policy Optimization (PPO) for the ALFWorld environment. Developed by chanyoungkim, this specific checkpoint represents optimizer step 10 of the reinforcement learning process. It is designed to improve agent performance in text-based interactive environments, demonstrating an in-training validation success rate of 7.8% at this stage.

Loading preview...

Model Overview

This model, chanyoungkim/ppo-qwen2.5-1.5b-alfworld-step10, is a 1.54 billion parameter Qwen/Qwen2.5-1.5B-Instruct backbone that has undergone Reinforcement Learning (RL) fine-tuning using the Proximal Policy Optimization (PPO) algorithm. The training specifically targets the ALFWorld environment (alfworld/AlfredTWEnv), aiming to enhance the model's ability to interact and solve tasks within text-based game settings.

Key Characteristics

  • Base Model: Qwen/Qwen2.5-1.5B-Instruct (1.54B parameters, bf16).
  • Fine-tuning Method: PPO, utilizing the langfengQ/verl-agent framework.
  • Target Environment: ALFWorld, a simulated interactive environment for embodied agents.
  • Checkpoint Specifics: This release corresponds to optimizer step 10 of the PPO training run.
  • Performance at Step 10: Achieved an in-training validation success rate of 7.8% on a 32-task validation batch.
  • Training Details: Uses specific hyperparameters for RL, including GAE for advantage estimation, KL loss, and an invalid action penalty. The critic learning rate is set at 1e-5.

Intended Use Cases

This model is particularly suited for:

  • Research and development in reinforcement learning for language models.
  • Experimentation with agent behavior in interactive text environments like ALFWorld.
  • Further fine-tuning or as a starting point for developing more capable agents for similar tasks.

It's important to note that this is an intermediate checkpoint, and later steps in the training run show significantly higher success rates, indicating ongoing learning and improvement.

Finetunes
1 models