Model Overview
g-ronimo/phi-2-OpenHermes-2.5 is a 3 billion parameter language model built upon Microsoft's phi-2 architecture, further fine-tuned with the teknium/OpenHermes-2.5 dataset. This fine-tuning process utilized QLoRA with a rank of 32, a learning rate of 2e-5, and was trained for 1 epoch with an effective batch size of 200 and a maximum sequence length of 1024 tokens.
Key Capabilities
- Instruction Following: The model is designed to follow instructions effectively, as demonstrated by its chat template inference example.
- Improved Benchmark Performance: It shows a slight improvement in AGIEval scores (30.27) compared to the base
phi-2 model (27.96) and a similar fine-tune by minghaowu (27.95), indicating enhanced reasoning capabilities. - Efficient Inference: With its 3 billion parameters, it offers a balance between performance and computational efficiency, suitable for various deployment scenarios.
Good For
- General Conversational AI: Its instruction-tuned nature makes it suitable for chatbots and interactive applications.
- Small-Scale Deployments: The model's size allows for more accessible deployment on devices with limited resources.
- Experimentation: Developers can use this model as a base for further fine-tuning on specific tasks due to its solid foundation and demonstrated performance improvements.