SingleBicycle/ccpo-alfworld-qwen2.5-7b-return-hard

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Sep 13, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

SingleBicycle/ccpo-alfworld-qwen2.5-7b-return-hard is a 7.6 billion parameter Qwen2.5-7B-Instruct model fine-tuned for agent policy optimization in text-based environments. It utilizes Context-Conditioned Policy Optimization (CCPO) on the ALFWorld benchmark, specifically targeting credit assignment in multi-turn agent reinforcement learning. This model is designed to improve an agent's ability to solve complex household tasks by learning direct solutions rather than extensive searching, achieving a held-out success rate of 83.5% on unseen ALFWorld tasks.

Loading preview...

Overview

This model, SingleBicycle/ccpo-alfworld-qwen2.5-7b-return-hard, is an interim checkpoint of a 7.6 billion parameter Qwen2.5-7B-Instruct base model. It has been fine-tuned using Context-Conditioned Policy Optimization (CCPO) on the ALFWorld text-based embodied household benchmark. CCPO is a method for credit assignment in multi-turn agent reinforcement learning, which refines how an agent attributes success or failure across a sequence of actions.

Key Capabilities & Features

  • Agent Policy Optimization: Specifically trained to generate effective policies for navigating and interacting within text-based environments like ALFWorld.
  • Context-Conditioned Policy Optimization (CCPO): Employs a novel approach to credit assignment by using a context-conditioned, uncertainty-shrunk leave-one-out baseline for the step term of the advantage function.
  • Performance on ALFWorld: Achieves a held-out success rate of 83.5% on unseen ALFWorld tasks (converged window, step >= 70), with a best single evaluation of 93.8% at step 115. The model significantly reduces the mean number of environment interactions per episode as it learns.
  • Base Model: Built upon the robust Qwen2.5-7B-Instruct architecture, providing strong language understanding and generation capabilities.

Distinguishing Characteristics

This specific arm of the CCPO training uses an exact (task, observation) anchor gate, uniform sibling weighting, and computes step credit on the discounted return-to-go. While it demonstrates strong performance, the developers note that these results are not directly comparable to published ALFWorld baselines (e.g., GRPO, GiGPO) which used a smaller Qwen2.5-1.5B-Instruct model. The primary focus of this model is to isolate and evaluate the CCPO estimator's effectiveness.

Usage

Developers can load the model and tokenizer using the Hugging Face transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "SingleBicycle/ccpo-alfworld-qwen2.5-7b-return-hard"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

The policy expects ALFWorld agent prompts and outputs actions within <action> tags.

Finetunes
2 models