EmbeddedLLM/Mistral-7B-Merge-14-v0.3-ft-step-15936

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Jan 5, 2024License:apache-2.0Architecture:Transformer Open Weights Cold

EmbeddedLLM/Mistral-7B-Merge-14-v0.3-ft-step-15936 is a 7 billion parameter language model fine-tuned from EmbeddedLLM/Mistral-7B-Merge-14-v0.3. This model has been fine-tuned for 3 epochs on a diverse dataset including dophin, dolphin-coder, Magicoder-OSS-Instruct-75K, openhermes, and Synthia-v1.3, suggesting a focus on general conversational and coding assistance. It utilizes a 4096-token context length and is designed for applications requiring a capable 7B model with broad instruction-following abilities.

Loading preview...

Overview

EmbeddedLLM/Mistral-7B-Merge-14-v0.3-ft-step-15936 is a 7 billion parameter language model, fine-tuned from the base model EmbeddedLLM/Mistral-7B-Merge-14-v0.3. The fine-tuning process involved 3 epochs of training using axolotl on a cluster of 4 A100 GPUs.

Key Capabilities

This model has been fine-tuned on a comprehensive set of datasets, indicating a broad range of instruction-following and conversational abilities. The training datasets include:

  • dophin
  • dolphin-coder
  • Magicoder-OSS-Instruct-75K
  • openhermes
  • Synthia-v1.3

The inclusion of datasets like dolphin-coder and Magicoder-OSS-Instruct-75K suggests enhanced capabilities in code generation and understanding, alongside general conversational proficiency from dophin, openhermes, and Synthia-v1.3.

Prompt Format

The model is designed to work with the ChatML prompt format, ensuring structured and clear interaction for both system and user inputs. An example of the expected format is:

<|im_start|>system
You are Dolphin, a helpful AI assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Good For

This model is suitable for developers seeking a 7B parameter model with a balanced instruction-following capability, particularly for tasks involving general chat, question answering, and potentially code-related assistance, given its diverse training data.

Finetunes
2 models