Blossom-v5-4b: A Conversational LLM

Blossom-v5-4b is a 4 billion parameter conversational large language model developed by Azure99, built upon the Qwen1.5-4B pre-trained model. This iteration, part of the Blossom V5 series, leverages high-quality data distilled from gpt-4-0125-preview to significantly improve its performance in conversational tasks and instruction following. It features a substantial context length of 32768 tokens.

Key Capabilities

• Enhanced Conversational Abilities: Fine-tuned on a mixed dataset including Blossom Orca, Wizard, Chat, and Math, it demonstrates strong general capabilities and context comprehension in dialogues.
• High-Quality Training Data: Benefits from a two-stage training process using 40K Wizard, 40K Orca, 10K Math single-turn instruction datasets, followed by 10K Blossom chat multi-turn dialogue data.
• Multilingual Support: Trained with high-quality Chinese and English datasets, which have also been open-sourced.
• Dialogue Continuation: Designed for effective single-turn and multi-turn dialogue inference, requiring specific formatting for historical conversations.

Good For

• Developing conversational AI agents and chatbots.
• Applications requiring robust instruction following and context understanding.
• Use cases involving both English and Chinese language interactions.