Hugging Face Overview
Overview
rhysjones/phi-2-orange-v2 is an improved, two-step fine-tuned version of Microsoft's Phi-2 model. It builds upon the original Phi-2-Orange by utilizing an updated training process on the same datasets and incorporating the latest Phi-2 model, ensuring direct compatibility with Hugging Face's Transformers library without the need for trust_remote_code.
Key Capabilities & Features
- Improved Fine-tuning: Benefits from an enhanced two-step fine-tuning process.
- Direct Hugging Face Compatibility: Seamlessly integrates with the Transformers library.
- ChatML Prompt Format: Employs the ChatML format for structured conversations, including system instructions for guiding model behavior (e.g., controlling verbosity or specifying output format).
Performance & Evaluations
Evaluations on the Open LLM Leaderboard show an Average score of 63.67, with specific metrics including:
- AI2 Reasoning Challenge (25-Shot): 61.86
- HellaSwag (10-Shot): 76.32
- MMLU (5-Shot): 55.72
- TruthfulQA (0-shot): 54.84
- Winogrande (5-shot): 75.69
- GSM8k (5-shot): 57.62
Additional evaluations on the YALL Leaderboard report an Average score of 49.64.
Limitations
This model inherits the limitations of the base Microsoft Phi-2 model, which are detailed in its original documentation.