jvonrad/Qwen-2.5-7B-grpo-bonus5-ablation

Model Overview

jvonrad/Qwen-2.5-7B-grpo-bonus5-ablation is a specialized variant of the Qwen2.5-7B large language model, featuring 7.6 billion parameters. This model has undergone fine-tuning using the GRPO (Generative Reinforcement Learning with Policy Optimization) method, incorporating a specific --all_correct_bonus 5.0 setting. This particular configuration deviates from the standard GRPO run by applying an increased bonus for entirely correct responses.

Key Capabilities and Training

The primary focus of this model's training is the WIKI-FACT / PolyFact cross-lingual factual-recall task. This indicates an optimization for retrieving and generating factually consistent information across different languages. The training methodology aims to improve the model's ability to maintain factual accuracy, especially in scenarios requiring multilingual understanding and generation.

Evaluation

Performance of this ablation model has been evaluated for consistency on the PolyFact dataset, as detailed in results/qwen-2.5-7b-grpo-bonus5-ablation_polyfact_consistency.json. Additionally, its consistency on a lite version of the GMMLU benchmark is available in results/qwen-2.5-7b-grpo-bonus5-ablation_gmmlu_lite_consistency.json. These evaluations provide insights into the model's factual recall and consistency capabilities under the specified GRPO bonus setting.

When to Use This Model

This model is particularly suited for use cases requiring:

• Enhanced factual consistency: Its GRPO fine-tuning with an all-correct bonus is designed to improve the reliability of generated facts.
• Cross-lingual factual recall: Optimized for tasks involving factual information retrieval and generation across multiple languages, leveraging the WIKI-FACT / PolyFact dataset.
• Research into GRPO ablation studies: Ideal for researchers exploring the impact of specific GRPO hyperparameters, such as the all_correct_bonus, on model performance in factual tasks.