Klingspor/StarPO-4B
Klingspor/StarPO-4B is a 4 billion parameter, 32768 context length language model based on Qwen3-4B, fine-tuned using StarPO (a GRPO variant) for the 20 Questions task. Developed by Klingspor, this model acts as a questioner agent, strategically asking yes-or-no questions to deduce a secret word. It is specifically optimized for multi-turn interactive language agent research and serves as a baseline for intrinsic credit assignment methods in multi-step reinforcement learning.
Loading preview...
Overview of Klingspor/StarPO-4B
Klingspor/StarPO-4B is a specialized 4 billion parameter language model, a reinforcement learning fine-tuned version of Qwen3-4B. Its core function is to act as a Questioner in the game of 20 Questions, where it asks strategic yes-or-no questions to identify a secret word. This model was developed as part of the paper "Intrinsic Credit Assignment for Long Horizon Interaction" and utilizes StarPO, a variant of Group Relative Policy Optimization (GRPO) adapted for multi-turn settings.
Key Capabilities
- Strategic Questioning: Designed to ask clear, concise, and deductive yes/no questions in a multi-turn interactive setting.
- Reinforcement Learning Fine-tuning: Leverages StarPO for improved performance in long-horizon interactive tasks.
- Research Baseline: Serves as a foundational model for studying intrinsic credit assignment in multi-step reinforcement learning for language agents.
Training Details
The model's training began from a Qwen3-4B-SFT checkpoint. It was trained on 1,000 words from the COCA+ RL training set, with a Qwen3-14B model acting as the judge/oracle. The training framework used was VERL.
Intended Use Cases
- 20 Questions Agent: Directly usable for playing the 20 Questions game as the questioner.
- RL Research: Ideal for research into multi-turn interactive language agents and reinforcement learning for large language models.
- Comparative Analysis: Suitable as a baseline for evaluating new credit assignment methods in multi-step RL scenarios.