Hugging Face Overview
Overview
Azure99/blossom-v5-14b is a conversational large language model built upon the Qwen1.5-14B pre-trained model. The Blossom V5 series, including this 14.2 billion parameter variant, has been significantly improved through training on high-quality data distilled from gpt-4-0125-preview. It is designed for robust general capabilities and strong context comprehension, particularly in dialogue scenarios.
Key Capabilities
- Conversational AI: Optimized for multi-turn dialogue and chat applications.
- Enhanced Comprehension: Benefits from training on data distilled from gpt-4-0125-preview, leading to improved understanding and response generation.
- Multilingual Support: Utilizes high-quality Chinese and English datasets, which have been open-sourced.
- Context Length: Supports a substantial context window of 32768 tokens, allowing for extended conversations.
Training Details
The model underwent a two-stage fine-tuning process:
- Stage 1: Trained for 1 epoch on 40K Wizard, 40K Orca, and 10K Math single-turn instruction datasets.
- Stage 2: Trained for 3 epochs on a 10K Blossom chat multi-turn dialogue dataset, combined with 10% randomly sampled data from the first stage.
Use Cases
This model is particularly well-suited for applications requiring:
- Interactive chatbots and virtual assistants.
- Dialogue systems that need to maintain context over multiple turns.
- General-purpose conversational AI tasks where strong comprehension and helpful responses are critical.