SeanWang0027/qwen3-1.7b-agentick-rose-s10t10-qwen3-32b-step300
SeanWang0027/qwen3-1.7b-agentick-rose-s10t10-qwen3-32b-step300 is a 1.7 billion parameter Qwen3-based student model, fine-tuned using an online multi-turn ROSE (Reinforcement Learning from Online Self-Correction) method. It was trained with a Qwen3-32B teacher on the Agentick gridworld environment, focusing on multi-turn agent capabilities. This model is specifically designed for sequential decision-making tasks within interactive environments, utilizing a ReAct-style prompt format for thought and action generation.
Loading preview...
Model Overview
This model, SeanWang0027/qwen3-1.7b-agentick-rose-s10t10-qwen3-32b-step300, is a 1.7 billion parameter student model based on the Qwen3 architecture. It is part of a research study investigating different training methodologies for multi-turn agents, specifically comparing teacher-trajectory SFT, online multi-turn ROSE, and on-policy distillation.
Training Methodology
The model was trained using an online multi-turn ROSE (Reinforcement Learning from Online Self-Correction) approach. During training, the student model plays the first 10 turns in a live environment, after which a Qwen3-32B teacher model continues for up to 10 additional turns. Only the teacher's turns are used as cross-entropy targets for the student. Training was conducted on the Agentick ms60 dataset for 300 steps, with 32 episodes per step, using AdamW optimizer and bf16 precision.
Key Capabilities
- Multi-turn Agent Behavior: Designed to handle sequential interactions and decision-making in dynamic environments.
- ReAct Prompting: Utilizes a ReAct-style prompt format, generating
Thought:andAction:pairs for each turn. - Teacher-Guided Learning: Benefits from a larger Qwen3-32B teacher model's trajectories for improved performance in complex tasks.
Evaluation Insights
Preliminary evaluations on the Agentick official eval set show that while the base Qwen3-1.7B achieved 5.70% success, the Qwen3-32B teacher reached 22.40%, and a teacher-SFT variant achieved 25.30%. This specific model's success rate was not explicitly provided in the one-pass evaluation, but it is positioned within the context of these comparative benchmarks.
Good For
- Agentic Applications: Ideal for tasks requiring an agent to interact with an environment over multiple turns.
- Sequential Decision-Making: Suitable for scenarios where a model needs to plan and execute actions based on observations.
- Research in Agent Training: A valuable resource for researchers exploring online reinforcement learning and teacher-student distillation methods for agents.