Hugging Face Deita Quality Scorer: Automatic Instruction Quality Annotation
The hkust-nlp/deita-quality-scorer is a specialized 7 billion parameter model developed by HKUST NLP, fine-tuned from Llama-1-13b-hf. It is a core component of the Deita project, an open-source initiative focused on Automatic Data Selection for instruction tuning in Large Language Models (LLMs).
Key Capabilities
- Automated Quality Scoring: The model excels at automatically assigning a quality score to instruction-response pairs, which is vital for curating high-quality SFT datasets.
- Data Selection for LLMs: By providing a quantitative measure of instruction quality, it facilitates the selection of optimal data for training and fine-tuning LLMs.
- English Language Focus: Primarily designed and optimized for processing and scoring English instruction-response data.
Usage and Integration
Developers can integrate this model using the provided Python code snippet, which demonstrates how to input an instruction and a response to receive a quality score. This score helps in filtering or prioritizing data based on its perceived quality. The model's utility is further highlighted by its role within the broader Deita collection, which includes other related models and datasets aimed at improving LLM alignment and performance through better data.
Research Context
The Deita project and this scorer are backed by research detailed in the paper "What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning" (arXiv:2312.15685), emphasizing its foundation in academic rigor for practical application.