Azure99/blossom-v3-mistral-7b

Blossom-v3-mistral-7b Overview

Blossom-v3-mistral-7b is a 7 billion parameter conversational large language model developed by Azure99. It is fine-tuned on the Mistral-7B-v0.1 base model using a proprietary mixed dataset comprising Blossom Orca, Wizard, Chat, and Math data. This model is designed to offer strong general capabilities and context understanding, making it suitable for various conversational AI applications.

Key Capabilities

• Conversational AI: Excels in multi-turn dialogue and instruction following, providing helpful, detailed, and polite responses.
• Mathematical Reasoning: Incorporates a dedicated 2K Blossom math reasoning dataset in its training, enhancing its ability to handle mathematical queries.
• Multilingual Support: Trained with high-quality Chinese and English datasets, though for primarily Chinese scenarios, blossom-v3-baichuan2-7b is recommended due to the Mistral base model's Chinese knowledge limitations.
• Robust Context Comprehension: Demonstrates strong understanding of conversational context, crucial for engaging interactions.

Training Details

The model underwent a two-stage training process:

1. Stage 1: 1 epoch of training on 100K Wizard and 100K Orca single-turn instruction datasets.
2. Stage 2: 3 epochs of training on a 2K Blossom math reasoning dataset, 50K Blossom chat multi-turn dialogue dataset, and a 1% random sample from the first stage's data.

Good For

• General-purpose chatbots and virtual assistants.
• Applications requiring mathematical problem-solving capabilities.
• English-centric conversational tasks where robust context handling is important.
• Developers looking for a Mistral-based model with enhanced conversational and reasoning fine-tuning.