Jinyang23/Seed-AlfWorld-3B
Jinyang23/Seed-AlfWorld-3B is a 3.1 billion parameter language model built upon Qwen2.5-3B-Instruct, specifically trained for text-based embodied interaction within the ALFWorld environment. It utilizes the SEED (Self-Evolving On-Policy Distillation) method, which involves hindsight-skill SFT and self-evolving on-policy distillation, to achieve high success rates in complex interactive tasks. This model excels at acting in simulated environments by internalizing skills learned during training without requiring additional prompts or analyzers at inference time.
Loading preview...
Overview
Jinyang23/Seed-AlfWorld-3B is a 3.1 billion parameter model, based on Qwen2.5-3B-Instruct, designed for text-based embodied interaction in the ALFWorld environment. It employs a novel training approach called SEED (Self-Evolving On-Policy Distillation), which involves two key stages:
- Hindsight-skill SFT: The model learns to summarize completed trajectories into concise, episode-level skills.
- Self-evolving on-policy distillation: During reinforcement learning, the policy both acts in the environment and analyzes its own collected trajectories, using generated skills to provide dense token-level supervision. This allows both the policy and its hindsight supervision to evolve together.
Crucially, at inference time, the model operates solely from standard interaction history, requiring no external analyzer, skill bank, or additional skill prompts.
Key Capabilities & Differentiators
- Self-evolving supervision: Policy updates enhance both action selection and the analyzer for generating future hindsight skills.
- On-policy skill generation: Supervision is directly derived from the states, actions, and failure modes encountered by the current policy.
- Dense token-level credit: Skill-induced log-probability shifts offer decision-specific guidance beyond simple terminal rewards.
- No inference overhead: The complex hindsight skills are internalized during training, ensuring efficient deployment.
Performance
SEED-AlfWorld-3B demonstrates strong performance in ALFWorld tasks. On the ALFWorld seen test split, it achieves a 91.8% macro-average success rate, outperforming baselines like GRPO (75.0%) and SDAR (84.4%). For the ALFWorld unseen split, it reaches an 86.2% macro-average, significantly higher than the matched GRPO baseline (70.9%).
Limitations
This model is specialized for ALFWorld-style text interaction and may not generalize reliably to unrelated environments or different prompting formats.