Hugging Face Overview
Blossom-v5-4b: A Conversational LLM
Blossom-v5-4b is a 4 billion parameter conversational large language model developed by Azure99, built upon the Qwen1.5-4B pre-trained model. This iteration, part of the Blossom V5 series, leverages high-quality data distilled from gpt-4-0125-preview to significantly improve its performance in conversational tasks and instruction following. It features a substantial context length of 32768 tokens.
Key Capabilities
- Enhanced Conversational Abilities: Fine-tuned on a mixed dataset including Blossom Orca, Wizard, Chat, and Math, it demonstrates strong general capabilities and context comprehension in dialogues.
- High-Quality Training Data: Benefits from a two-stage training process using 40K Wizard, 40K Orca, 10K Math single-turn instruction datasets, followed by 10K Blossom chat multi-turn dialogue data.
- Multilingual Support: Trained with high-quality Chinese and English datasets, which have also been open-sourced.
- Dialogue Continuation: Designed for effective single-turn and multi-turn dialogue inference, requiring specific formatting for historical conversations.
Good For
- Developing conversational AI agents and chatbots.
- Applications requiring robust instruction following and context understanding.
- Use cases involving both English and Chinese language interactions.