LongWriter-Zero-32B: Ultra-Long Text Generation

LongWriter-Zero-32B, developed by THU-KEG, is a 32 billion parameter large language model built on Qwen 2.5-32B-Base. Its core innovation lies in its ability to generate coherent text passages exceeding 10,000 tokens, a capability achieved through a unique training methodology.

Key Capabilities & Training:

• Ultra-Long Text Generation: Specifically engineered to produce extended, coherent outputs, surpassing the length capabilities of many larger models.
• Reinforcement Learning (RL) Based: Utilizes Group Relative Policy Optimization (GRPO) with a sophisticated composite reward function, including length, writing quality, and format adherence, to guide generation.
• Continual Pretraining: Underwent 30 billion-token continual pretraining on long-form books and technical reports to bolster its foundational writing skills.
• Structured Output: Employs a dedicated prompting strategy that encourages explicit reflection before answering, improving structural planning and fine-grained length control, often using a <think>…</think><answer>…</answer> format.

Performance & Evaluation:

• Benchmark Leader: Achieves the highest automatic WritingBench score among open models.
• Human Evaluation: Demonstrates dominant win-rates in pairwise GPT-4.1 evaluations, confirming its superior quality in ultra-long-form generation.

Good for:

• Generating very long articles, reports, or creative narratives.
• Tasks requiring structured and coherent extended text outputs.
• Applications where fine-grained control over output length and format is crucial.