Azure99/blossom-v4-yi-34b

TEXT GENERATIONConcurrency Cost:2Model Size:34BQuant:FP8Ctx Length:32kPublished:Jan 2, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

Azure99/blossom-v4-yi-34b is a 34 billion parameter conversational large language model, fine-tuned by Azure99 on a mixed dataset including Orca, Wizard, Chat, and Math data, based on the Yi-34B pre-trained model. It is designed for robust general capabilities and strong context comprehension, excelling in both English and Chinese conversational tasks. The model's training focused on both single-turn instructions and multi-turn dialogues to enhance its interactive performance.

Loading preview...

Model Overview

Azure99/blossom-v4-yi-34b is a 34 billion parameter conversational large language model developed by Azure99. It is built upon the Yi-34B pre-trained model and has been extensively fine-tuned to enhance its dialogue capabilities and general comprehension. The model leverages a diverse dataset, including Orca, Wizard, Chat, and Math data, with a strong emphasis on both English and Chinese language processing.

Key Capabilities

  • Conversational AI: Optimized for engaging in natural and coherent multi-turn dialogues.
  • Context Comprehension: Demonstrates robust understanding of conversational context.
  • Multilingual Support: Trained on high-quality Chinese and English datasets, making it suitable for bilingual applications.
  • Instruction Following: Capable of processing and responding to single-turn instructions effectively.

Training Methodology

The training process for Blossom-v4-yi-34b involved two distinct stages:

  1. Stage One: Initial training on 100K Wizard, 100K Orca, and 20K Math single-turn instruction datasets for one epoch.
  2. Stage Two: Further training for three epochs using a 50K Blossom chat multi-turn dialogue dataset, supplemented with a 2% random sample from the first stage's data.

Use Cases

This model is particularly well-suited for applications requiring advanced conversational abilities, such as chatbots, virtual assistants, and interactive content generation, especially in environments where strong context understanding and bilingual support (English/Chinese) are beneficial.

Finetunes
4 models