Azure99/blossom-v4-mistral-7b

Blossom-v4-mistral-7b Overview

Blossom-v4-mistral-7b is a 7 billion parameter conversational large language model developed by Azure99. It is fine-tuned on the Mistral-7B-v0.1 base model using a unique mixed dataset comprising Blossom Orca, Wizard, Chat, and Math data. This two-stage training process involved initial instruction-following datasets (Wizard, Orca, Math) followed by multi-turn dialogue data (Blossom chat).

Key Capabilities

• Conversational AI: Designed for dialogue continuation, handling both single-turn and multi-turn interactions effectively.
• General Comprehension: Possesses robust general capabilities and strong context understanding due to its diverse training data.
• English Optimization: Primarily optimized for English language scenarios, leveraging the Mistral-7B-v0.1 base model's strengths.

Training Details

The model underwent a two-stage fine-tuning process:

• Stage 1: Trained for 1 epoch on 100K Wizard, 100K Orca, and 20K Math single-turn instruction datasets.
• Stage 2: Trained for 3 epochs on 50K Blossom chat multi-turn dialogue dataset, supplemented with 2% randomly sampled data from Stage 1.

Usage Notes

• Inference is structured as dialogue continuation, requiring specific formatting for single and multi-turn conversations.
• For Chinese language applications, the developers recommend using blossom-v4-baichuan2-7b due to the Mistral-7B-v0.1 base model's limitations in Chinese knowledge.