yuhan-nlp/verl-grpo-medium-qwen3-4b-step100

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:4BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 9, 2026License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

The yuhan-nlp/verl-grpo-medium-qwen3-4b-step100 is a 4 billion parameter Qwen3-based language model developed by yuhan-nlp, fine-tuned using GRPO (Generative Reinforcement Learning with Policy Optimization) for the CollabLLM medium document-writing task. This model, specifically checkpoint step 100, demonstrates the best BLEU score among its GRPO-trained counterparts, significantly outperforming the base Qwen3-4B model in document quality. It is optimized for collaborative document generation, focusing on improving text coherence and relevance.

Loading preview...

Model Overview

yuhan-nlp/verl-grpo-medium-qwen3-4b-step100 is a 4 billion parameter model based on the Qwen/Qwen3-4B architecture. It has been fine-tuned using Generative Reinforcement Learning with Policy Optimization (GRPO) within the verl framework for the CollabLLM medium document-writing task. This specific checkpoint (step 100) is highlighted as the most effective, achieving the highest BLEU score among the GRPO-trained arms.

Key Capabilities & Performance

  • Enhanced Document Quality (BLEU Score): This model achieves a BLEU score of 0.4827 on the CollabLLM medium benchmark, which is a notable improvement over the base Qwen3-4B's 0.4491. This indicates superior text generation quality in collaborative writing scenarios.
  • Optimized for Collaborative Writing: Trained with a reward function that includes bleu_score, interactivity, and token_amount, it is designed to produce high-quality, concise contributions in a multi-agent document creation environment.
  • Efficient Generation: While prioritizing BLEU, the model also maintains reasonable interactivity and token usage, with an average of 2.077k tokens per interaction.

Important Considerations

  • Checkpoint Selection: Training peaked at step 100; later steps (e.g., step 129) showed regression in BLEU despite increases in the internal critic/rewards/mean metric. Users should rely on external metrics like BLEU for evaluation.
  • Metric Focus: While this model excels in BLEU, prompting-alone strategies might still yield higher MR (Mean Reciprocal Rank) scores, particularly when token reduction is a primary goal. The MR gain in GRPO-trained models is largely influenced by token reduction rather than document quality alone.
  • Data Leakage Warning: The benchmark includes a 35/100 train/eval prompt overlap with byte-identical reward targets, indicating potential label leakage through the reward mechanism. Refer to the dataset card for more details.

Finetunes
2 models