yapeichang/Qwen2.5-7B-RM8B
The yapeichang/Qwen2.5-7B-RM8B model is a 7.6 billion parameter language model based on the Qwen2.5-7B architecture, developed by Yapei Chang and collaborators. It is specifically fine-tuned using GRPO with the Skywork-RM-8B reward model, focusing on general instruction following. This model demonstrates performance comparable to RM-guided GRPO across various instruction-following benchmarks, producing factually grounded outputs.
Loading preview...
Overview
This model, yapeichang/Qwen2.5-7B-RM8B, is a 7.6 billion parameter language model built upon the Qwen2.5-7B architecture. It was developed by Yapei Chang and a team of researchers, as detailed in their paper "BLEUBERI: BLEU is a surprisingly effective reward for instruction following." The model is fine-tuned using a novel approach called GRPO (Generalized Reward Policy Optimization) where BLEU, a simple n-gram matching metric, is directly used as a reward signal.
Key Capabilities
- Enhanced Instruction Following: The model is specifically optimized for general instruction following tasks, extending beyond easily verifiable domains like math and code.
- Factually Grounded Outputs: It is designed to produce more factually grounded responses, with human evaluations rating its outputs on par with systems trained using traditional reward models.
- GRPO Training with BLEU: Utilizes a unique training methodology where BLEU, paired with high-quality references from strong LLMs, serves as an effective reward function.
Good For
- Applications requiring robust general instruction following capabilities.
- Generating responses that are both coherent and factually consistent.
- Research and development in reward modeling and reinforcement learning from human feedback (RLHF) alternatives.