yapeichang/Llama-3.1-8B-BLEUBERI

TEXT GENERATIONPricing:Input $0.2 / Cached $0.028 / Output $0.32Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:May 27, 2025License:llama3.1Architecture:Transformer0.0K Featherless Exclusive Cold

yapeichang/Llama-3.1-8B-BLEUBERI is an 8 billion parameter Llama-3.1 based language model developed by Yapei Chang and colleagues, fine-tuned using BLEU as a direct reward in GRPO training. This approach extends Reinforcement Learning from Verifiable Rewards (RLVR) to general instruction following, achieving performance comparable to reward model-guided GRPO across four instruction-following benchmarks. It is designed to produce more factually grounded outputs, matching human ratings of reward model-trained systems.

Loading preview...

Model Overview

yapeichang/Llama-3.1-8B-BLEUBERI is an 8 billion parameter language model built upon the Llama-3.1 architecture. Developed by Yapei Chang and a team of researchers, this model introduces a novel training methodology called BLEUBERI, which utilizes the BLEU n-gram matching metric directly as a reward signal within GRPO (Generative Reinforcement Learning with Policy Optimization) training.

Key Differentiators

  • BLEU as Reward: Unlike traditional methods that rely on complex reward models, BLEUBERI leverages BLEU scores, paired with high-quality references from strong LLMs, to guide reinforcement learning. This simple yet effective approach extends Reinforcement Learning from Verifiable Rewards (RLVR) to more open-ended instruction-following tasks.
  • Performance Parity: The model demonstrates performance on par with systems trained using reward model-guided GRPO across four distinct general instruction-following benchmarks.
  • Factual Grounding: BLEUBERI is noted for producing more factually grounded outputs, with human evaluators rating its responses similarly to those from reward model-trained systems.

Use Cases

  • General Instruction Following: Excels in tasks requiring adherence to diverse instructions, making it suitable for a wide range of conversational and generative AI applications.
  • Factually Grounded Generation: Ideal for scenarios where factual accuracy and reduced hallucination are critical, such as information retrieval, summarization, or question answering.
  • Research in RLHF Alternatives: Offers a compelling alternative to traditional Reward Model (RM) based Reinforcement Learning from Human Feedback (RLHF) approaches, particularly for researchers exploring simpler, yet effective, reward mechanisms.

Finetunes
1 models