Hugging Face Overview
YanoljaNEXT-EEVE-Instruct-2.8B is an instruction-tuned large language model with 2.8 billion parameters, developed by Yanolja. It is built upon the yanolja/EEVE-Korean-2.8B-v1.0 base model, which itself is a Korean vocabulary-extended version of microsoft/phi-2. The instruction tuning was performed using Direct Preference Optimization (DPO) via the Axolotl framework.
Key Capabilities & Training
This model is specifically optimized for conversational interactions in Korean. Its training involved Korean-translated versions of well-known datasets:
- Open-Orca/SlimOrca-Dedup: A deduplicated subset of the SlimOrca dataset.
- argilla/ultrafeedback-binarized-preferences-cleaned: A dataset focused on binarized preferences for feedback.
The model's prompt template is structured for chat-based interactions, expecting a "Human: {prompt}\nAssistant:" format to generate helpful, detailed, and polite responses. Further technical details on its vocabulary expansion are available in the paper "Efficient and Effective Vocabulary Expansion Towards Multilingual Large Language Models".
Performance
On the Open LLM Leaderboard, the model achieved an average score of 58.71. Specific benchmark results include:
- AI2 Reasoning Challenge (25-Shot): 58.28
- HellaSwag (10-Shot): 72.42
- MMLU (5-Shot): 53.35
- TruthfulQA (0-shot): 48.32
- Winogrande (5-shot): 74.82
- GSM8k (5-shot): 45.11
Use Cases
This model is particularly well-suited for applications requiring instruction-following and conversational AI in the Korean language, such as chatbots, virtual assistants, or content generation tools that need to provide detailed and polite responses.