Azure99/blossom-v4-qwen1_5-7b

Blossom-v4-qwen1_5-7b Overview

Blossom-v4-qwen1_5-7b is a 7.7 billion parameter conversational language model developed by Azure99. It is built upon the Qwen1.5-7B pre-trained model and has undergone a two-stage instruction-tuning process. The training leverages a unique hybrid dataset, including Blossom Orca, Wizard, Chat, and Math data, which contributes to its robust general capabilities and strong contextual understanding.

Key Capabilities

• Conversational AI: Specifically fine-tuned for multi-turn dialogue, enabling natural and coherent interactions.
• Multilingual Support: Trained on high-quality Chinese and English datasets, making it proficient in both languages.
• Instruction Following: Demonstrates strong ability to follow instructions across various tasks due to its diverse training data.
• Context Understanding: Designed to maintain context effectively throughout extended conversations.

Training Methodology

The model's training involved two distinct phases:

1. Phase 1: Initial training for one epoch using 100K Wizard, 100K Orca, and 20K Math single-turn instruction datasets.
2. Phase 2: Further training for three epochs using 50K Blossom chat multi-turn dialogue datasets, supplemented by a 2% random sample from the first phase's data.

Good For

• General-purpose chatbots and conversational agents.
• Applications requiring strong Chinese and English language processing.
• Tasks benefiting from robust instruction following and context retention in dialogue.