SeanWang0027/qwen3-1.7b-textcraft-opd-qwen3-32b-step33

Model Overview

This model, SeanWang0027/qwen3-1.7b-textcraft-opd-qwen3-32b-step33, is a 1.7 billion parameter student model based on the Qwen/Qwen3-1.7B architecture. It was developed as part of a research study comparing teacher-trajectory SFT, online multi-turn ROSE, and on-policy distillation (OPD) for multi-turn agents.

Training Details

The model was trained using multi-turn on-policy distillation (OPD). During training, the student model played full episodes of 30 turns and was guided by a reverse-KL policy-gradient signal derived from the Qwen3-32B teacher model. Training focused on the student's own generated tokens, utilizing a learning rate of 1e-6, weight decay of 0.1, and a batch size of 32. The process involved 6 epochs over 374 tasks, totaling 66 steps, and was conducted in bf16 precision.

Evaluation Insights

Evaluation was performed on the TextCraft official test split (100 tasks), averaging 4 passes with a temperature of 0.4, 512 tokens per turn, and 30 turns. While a 'Success' metric was not explicitly evaluated, reference points indicate the base Qwen3-1.7B achieved 23.00% and the teacher Qwen3-32B achieved 85.50% on the same protocol.

Prompt Format

The model expects a ReAct, multi-turn chat prompt format. This includes the environment's AgentGym instruction as the initial user turn, followed by a canned assistant acknowledgment, and then one user turn per observation. The model's responses are structured as Thought:\n...\n\nAction:\n<one action>, rendered using Qwen3's chat template with enable_thinking=False.