Overview

rhysjones/phi-2-orange-v2 is an improved, two-step fine-tuned version of Microsoft's Phi-2 model. It builds upon the original Phi-2-Orange by utilizing an updated training process on the same datasets and incorporating the latest Phi-2 model, ensuring direct compatibility with Hugging Face's Transformers library without the need for trust_remote_code.

Key Capabilities & Features

• Improved Fine-tuning: Benefits from an enhanced two-step fine-tuning process.
• Direct Hugging Face Compatibility: Seamlessly integrates with the Transformers library.
• ChatML Prompt Format: Employs the ChatML format for structured conversations, including system instructions for guiding model behavior (e.g., controlling verbosity or specifying output format).

Performance & Evaluations

Evaluations on the Open LLM Leaderboard show an Average score of 63.67, with specific metrics including:

• AI2 Reasoning Challenge (25-Shot): 61.86
• HellaSwag (10-Shot): 76.32
• MMLU (5-Shot): 55.72
• TruthfulQA (0-shot): 54.84
• Winogrande (5-shot): 75.69
• GSM8k (5-shot): 57.62

Additional evaluations on the YALL Leaderboard report an Average score of 49.64.

Limitations

This model inherits the limitations of the base Microsoft Phi-2 model, which are detailed in its original documentation.