chanyoungkim/grpo-qwen2.5-1.5b-alfworld-step140

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 7, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

This model is chanyoungkim/grpo-qwen2.5-1.5b-alfworld-step140, an RL fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct, featuring 1.54 billion parameters and a 32768-token context length. It is specifically optimized for interactive decision-making tasks within the ALFWorld environment using the GRPO algorithm. This checkpoint, at optimizer step 140, demonstrates a 66.4% in-training validation success rate, making it suitable for research and development in reinforcement learning for text-based environments.

Loading preview...

Overview

This model, chanyoungkim/grpo-qwen2.5-1.5b-alfworld-step140, is a Reinforcement Learning (RL) fine-tuned version of the Qwen/Qwen2.5-1.5B-Instruct base model. It utilizes the GRPO (Generalized Reinforcement Learning with Policy Optimization) algorithm, specifically trained on the ALFWorld environment for interactive text-based tasks. This particular checkpoint represents the model at optimizer step 140.

Key Capabilities & Performance

  • RL Fine-tuning: Optimized for sequential decision-making in interactive environments using GRPO.
  • ALFWorld Specialization: Demonstrates proficiency in the ALFWorld environment, achieving an in-training validation success rate of 66.4% at this step.
  • Base Model: Built upon the 1.54 billion parameter Qwen2.5-1.5B-Instruct, leveraging its foundational language understanding.
  • Training Details: Trained using 2x NVIDIA A100 80GB GPUs, with specific hyperparameters for RL, including a learning rate of 1e-6 and a ppo_mini_batch_size of 256.

Good For

  • Research in RL for LLMs: Ideal for researchers exploring GRPO and similar RL algorithms for fine-tuning language models in interactive settings.
  • ALFWorld Benchmarking: Useful for evaluating and comparing performance on ALFWorld tasks, particularly for models fine-tuned with GRPO.
  • Resuming RL Training: The provided training_state/ allows for resuming RL training from this exact optimizer step, though it's sharded for world_size=2.

Limitations

  • Validation was performed on ALFWorld valid_seen only; valid_unseen was not evaluated.
  • Success rates are based on in-training validation and may not directly compare to other benchmarks due to differing evaluation setups (e.g., StraTA reproduction uses a different agent scaffold and evaluation parameters).

Finetunes
1 models