arcee-ai/Arcee-VyLinh
Arcee-VyLinh is a 3 billion parameter instruction-following model developed by arcee-ai, built upon the Qwen2.5-3B architecture with a 32K token context length. It is specifically optimized for exceptional performance in Vietnamese language understanding and generation, utilizing an innovative training process including evolved hard questions and iterative Direct Preference Optimization (DPO). This compact model excels at complex Vietnamese language tasks, making it suitable for chat, text generation, and general language understanding in Vietnamese.
Loading preview...
Arcee-VyLinh: A Compact, High-Performance Vietnamese LLM
Arcee-VyLinh, developed by arcee-ai, is a 3 billion parameter instruction-following model based on the Qwen2.5-3B architecture, featuring a 32K token context length. It stands out for its specialized optimization for the Vietnamese language, achieving strong performance comparable to larger 4B-8B parameter models despite its compact size.
Key Capabilities & Training Innovations
- Vietnamese Language Mastery: Engineered for exceptional performance on complex Vietnamese language tasks, including chat, text generation, and question answering.
- Efficient Architecture: Leverages a 3B parameter count for efficient deployment while maintaining strong instruction-following capabilities.
- Advanced Training: Utilizes a multi-stage training process that includes:
- Starting with Qwen2.5-3B as the base.
- Generating 20K challenging questions using EvolKit for hard question evolution.
- Supervised fine-tuning (SFT) to create VyLinh-SFT.
- Proprietary model merging techniques.
- Six epochs of iterative Direct Preference Optimization (DPO) using the ORPO-Mix-40K (Vietnamese) dataset.
- Benchmark Performance: Demonstrates competitive results on the Vietnamese subset of m-ArenaHard (CohereForAI), as judged by Claude 3.5 Sonnet.
Intended Use Cases
- Vietnamese language chat and instruction following.
- Text generation and completion in Vietnamese.
- Question answering for Vietnamese content.
- General language understanding tasks in Vietnamese.
- Content creation and summarization in Vietnamese.
Limitations
While highly capable, Arcee-VyLinh's primary focus is on Vietnamese language, and it may not perform optimally for highly specialized technical domains or exhibit occasional hallucinations on cultural-specific content.