simonts/genre2-grm-sft
The simonts/genre2-grm-sft model is a 7 billion parameter supervised fine-tuned generative reward model, based on Mistral-7B-Instruct-v0.2, designed as a local GRM checkpoint candidate for GenRe2 experiments. This model is specifically optimized for reward modeling within a generative context, aiming to evaluate and guide text generation. It processes inputs with a 4096 token context length and is released under the Apache-2.0 license.
Loading preview...
GenRe2 GRM SFT Checkpoint Overview
The simonts/genre2-grm-sft model is a 7 billion parameter supervised fine-tuned (SFT) generative reward model (GRM). It serves as a local checkpoint candidate for the GenRe2 experiments, building upon the mistralai/Mistral-7B-Instruct-v0.2 base architecture. This model is specifically developed to function as a reward signal within generative AI systems.
Key Characteristics
- Base Model: Fine-tuned from
mistralai/Mistral-7B-Instruct-v0.2. - Purpose: Designed as a generative reward model (GRM) for the GenRe2 experimental framework.
- Context Length: Supports a context window of 4096 tokens.
- License: Released under the Apache-2.0 license, consistent with its base model.
- Size: The checkpoint is approximately 14 GB locally.
Performance Insights
During release preparation, the model's performance was evaluated against several benchmarks, with the Mistral-7B-Instruct-v0.2-sft_epoch2 variant showing an average score of 0.4471 across FB Bench, FLASK, MT Bench, and Vicuna benchmarks. This result is noted to be close to the paper's CE average of 0.4494, indicating its alignment with the intended experimental performance targets.
Use Cases
This model is primarily intended for:
- Research in Generative Reward Modeling: Ideal for experiments and studies focused on using generative models as reward signals.
- Evaluation of Text Generation: Can be integrated into systems to provide a learned reward for generated text, guiding further optimization or selection.
- Development of GenRe2-like Systems: Suitable for developers and researchers working on frameworks similar to GenRe2 that require a robust GRM component.