yuhan-nlp/verl-grpo-medium-qwen3-4b-step50

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:4BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 9, 2026License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

The yuhan-nlp/verl-grpo-medium-qwen3-4b-step50 is a 4 billion parameter Qwen3-based language model, developed by yuhan-nlp, fine-tuned using GRPO on the CollabLLM medium document-writing task. This intermediate checkpoint, with a 32768 token context length, shows early improvements in token reduction while maintaining BLEU scores, indicating optimization for efficient document generation in collaborative writing scenarios.

Loading preview...

Model Overview

This model, yuhan-nlp/verl-grpo-medium-qwen3-4b-step50, is a 4 billion parameter variant of the Qwen/Qwen3-4B architecture. It has been fine-tuned using the GRPO (Generative Reinforcement Learning with Policy Optimization) method on the CollabLLM medium document-writing task. This specific checkpoint represents an intermediate training stage (step 50) and is provided as a merged Hugging Face checkpoint, requiring no additional adapter steps.

Key Characteristics

  • Architecture: Based on Qwen/Qwen3-4B.
  • Training Method: Utilizes GRPO for optimization on a document-writing task.
  • Performance at Step 50: Benchmarks indicate that at step 50, the model's BLEU score is essentially flat compared to the base model, but it already shows a significant drop in token usage (2.281k vs 2.935k for base), suggesting improved brevity.
  • Context Length: Supports a context length of 32768 tokens.

Important Considerations

  • Training Peak: The README notes that training for this specific task peaked at step 100, with subsequent regression. This step 50 model is an intermediate point.
  • Metric Nuances: While this model shows token reduction, the README advises careful interpretation of metrics, noting that prompting alone can sometimes achieve better MR (Mean Reciprocal Rank) gains, and that MR gains are often dominated by token reduction rather than document quality.
  • Data Leakage Warning: The training setup involved a 35/100 train/eval prompt overlap with byte-identical reward targets, leading to label leakage through the reward mechanism for BLEU scores.

Finetunes
2 models