kixlab/prefmatcher-7b
The kixlab/prefmatcher-7b is a 7.6 billion parameter language model developed by kixlab, fine-tuned from Qwen2.5-7B-Instruct. It instantiates the Preference Match metric from the CUPID benchmark, designed to assess whether evaluation checklist items match a given preference description. This model provides a high-fidelity, cost-efficient solution for automatic evaluation in preference matching tasks, achieving a Krippendorff's alpha of 0.748 with human annotations.
Loading preview...
Overview
kixlab/prefmatcher-7b is a 7.6 billion parameter language model, fine-tuned from Qwen2.5-7B-Instruct, specifically designed to implement the Preference Match metric from the CUPID benchmark. Its core function is to determine if individual items within an evaluation checklist are 'covered' by a given preference description, meaning they evaluate similar aspects of AI outputs.
Key Capabilities
- Preference Matching: Assesses the alignment between a preference description and an evaluation checklist.
- Automatic Evaluation: Provides a high-fidelity and cost-efficient method for evaluating AI outputs based on specified preferences.
- Benchmarking: Serves as a judge for the CUPID benchmark, which focuses on personalized and contextualized alignment of LLMs.
Training Details
The model was fine-tuned using QLoRA for one epoch on 4,000 data samples, which were generated through a synthesis pipeline and evaluated by GPT-4o. It achieved a Krippendorff's alpha of 0.748 when compared to human annotations, indicating strong agreement. The training utilized the torchtune library.
Good For
- Developers and researchers working on automatic evaluation of LLM outputs.
- Implementing preference-based assessment systems.
- Analyzing the alignment of AI outputs with specific criteria or preferences.