Hugging Face Quyen-Mini-v0.1 Overview
Quyen-Mini-v0.1 is a 1.8 billion parameter model from the Quyen LLM series developed by vilm, built upon the Qwen1.5 family architecture. This model is one of six versions, ranging from 0.5B to 72B parameters, all trained with a focus on instruction following and preference alignment.
Key Capabilities & Training
The Quyen models, including Quyen-Mini-v0.1, underwent Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). The training utilized a comprehensive dataset comprising:
- OpenHermes-2.5 by Teknium
- Capyabara by LDJ
- argilla/distilabel-capybara-dpo-7k-binarized by argilla
- orca_dpo_pairs by Intel
- Proprietary data from Ontocord & BEE-spoke-data
All Quyen models use the ChatML prompt template, ensuring consistent and effective interaction for conversational AI applications.
Performance & Benchmarks
While detailed benchmarks are still being compiled, initial evaluations on the Open LLM Leaderboard show an average score of 46.14. Specific metric scores include:
- AI2 Reasoning Challenge (25-Shot): 39.33
- HellaSwag (10-Shot): 60.57
- MMLU (5-Shot): 43.93
- TruthfulQA (0-shot): 46.44
- Winogrande (5-shot): 59.12
- GSM8k (5-shot): 27.45
Good For
- General-purpose conversational agents: Its training on diverse instruction-following datasets makes it suitable for various dialogue tasks.
- Applications requiring a compact yet capable model: At 1.8B parameters, it offers a balance between performance and resource efficiency.
- Developers familiar with ChatML: The standardized prompt template simplifies integration into existing workflows.