Model Overview

The g-ronimo/phi-2-OpenHermes-2.5-v2 is a 3 billion parameter language model built upon Microsoft's phi-2 architecture. It has been further fine-tuned using the teknium/OpenHermes-2.5 dataset, specifically employing QLoRA with a rank of 32, a learning rate of 5e-5, and trained for 3 epochs with a batch size of 200 and a maximum sequence length of 1024 tokens.

Key Enhancements & Performance

This v2 iteration shows notable improvements over previous versions, including the base phi-2 and phi-2-OpenHermes-2.5. Evaluation benchmarks highlight its stronger performance:

• AGIEval: Achieves 32.02, an increase from 30.27 in phi-2-OpenHermes-2.5 and 27.96 in phi-2.
• Bigbench: Scores 36.85, up from 35.9 and 35.17 in earlier versions.
• Overall Average: Reaches 46.05 across AGIEval, GPT4All, TruthfulQA, and Bigbench.

Use Cases

This model is well-suited for applications requiring efficient instruction following and conversational generation, particularly where a balance between model size and performance is critical. Its enhanced reasoning capabilities make it a strong candidate for tasks involving complex queries and nuanced responses.