Azure99/blossom-v5.1-34b
Azure99/blossom-v5.1-34b is a 34 billion parameter conversational large language model, fine-tuned by Azure99 on a mixed dataset including Orca, Wizard, Chat, and Math data, based on the Yi-1.5-34B pre-trained model. It features robust general capabilities and strong context comprehension, with a 32768 token context length. This model is optimized for multi-turn dialogue and conversational AI applications, leveraging high-quality Chinese and English datasets.
Loading preview...
Blossom-v5.1-34b Overview
Blossom-v5.1-34b is a 34 billion parameter conversational large language model developed by Azure99, built upon the Yi-1.5-34B pre-trained architecture. It is specifically fine-tuned for dialogue, demonstrating robust general capabilities and strong context comprehension across various conversational tasks. The model's training incorporated a unique two-stage process using a mixed dataset, including Orca, Wizard, Chat, and Math instructions, with a focus on both single-turn and multi-turn dialogues.
Key Capabilities
- Conversational AI: Optimized for engaging in both single-turn and multi-turn dialogues.
- General Comprehension: Exhibits strong understanding across a broad range of topics.
- Multilingual Support: Trained on high-quality Chinese and English datasets, which have been open-sourced.
- Context Handling: Supports a substantial context length of 32768 tokens, enabling more coherent and extended conversations.
Training Methodology
The model underwent a two-stage fine-tuning process:
- Stage 1: Trained for 1 epoch on 40K Wizard, 40K Orca, and 10K Math single-turn instruction datasets.
- Stage 2: Trained for 3 epochs on a 10K Blossom chat multi-turn dialogue dataset, combined with 10% randomly sampled data from the first stage.
Ideal Use Cases
This model is particularly well-suited for applications requiring advanced conversational abilities, such as chatbots, virtual assistants, and interactive dialogue systems where robust general understanding and multi-turn coherence are critical.