Overview

Azure99/blossom-v5-14b is a conversational large language model built upon the Qwen1.5-14B pre-trained model. The Blossom V5 series, including this 14.2 billion parameter variant, has been significantly improved through training on high-quality data distilled from gpt-4-0125-preview. It is designed for robust general capabilities and strong context comprehension, particularly in dialogue scenarios.

Key Capabilities

• Conversational AI: Optimized for multi-turn dialogue and chat applications.
• Enhanced Comprehension: Benefits from training on data distilled from gpt-4-0125-preview, leading to improved understanding and response generation.
• Multilingual Support: Utilizes high-quality Chinese and English datasets, which have been open-sourced.
• Context Length: Supports a substantial context window of 32768 tokens, allowing for extended conversations.

Training Details

The model underwent a two-stage fine-tuning process:

1. Stage 1: Trained for 1 epoch on 40K Wizard, 40K Orca, and 10K Math single-turn instruction datasets.
2. Stage 2: Trained for 3 epochs on a 10K Blossom chat multi-turn dialogue dataset, combined with 10% randomly sampled data from the first stage.

Use Cases

This model is particularly well-suited for applications requiring:

• Interactive chatbots and virtual assistants.
• Dialogue systems that need to maintain context over multiple turns.
• General-purpose conversational AI tasks where strong comprehension and helpful responses are critical.