Overview

This model, EmbeddedLLM/Mistral-7B-Merge-14-v0.3-ft-step-9984, is a 7 billion parameter language model derived from EmbeddedLLM/Mistral-7B-Merge-14-v0.3. It has been extensively fine-tuned over 9984 steps to enhance its capabilities across various domains.

Key Characteristics

• Base Model: Fine-tuned from EmbeddedLLM/Mistral-7B-Merge-14-v0.3.
• Training Data: Utilizes a comprehensive dataset blend including dophin, dolphin-coder, Magicoder-OSS-Instruct-75K, openhermes, and Synthia-v1.3.
• Context Length: Supports a 4096 token context window.
• Prompt Format: Employs the ChatML format, with a specific structure for system, user, and assistant turns.
• Training Process: Fine-tuned for 3 epochs on 4 A100 GPUs using the axolotl framework.

Intended Use Cases

This model is well-suited for applications requiring:

• General Conversational AI: Its diverse training data suggests proficiency in various dialogue scenarios.
• Coding Assistance: The inclusion of datasets like dolphin-coder and Magicoder-OSS-Instruct-75K indicates strong performance in code generation, explanation, and debugging tasks.
• Instruction Following: Fine-tuning with instruction-based datasets enhances its ability to follow complex commands and generate relevant responses.