Dolphin 2.8 Experiment26 7b Overview

dphn/dolphin-2.8-experiment26-7b is a 7 billion parameter language model, fine-tuned from Experiment-26 by Yam Peleg. This iteration of Dolphin is notably optimized for coding tasks, having been trained with a significant volume of coding-specific data. The base model supports a 16k context, though this specific variant is listed with a 4096 token context length.

Key Training Details

• Training Method: Fine-tuned using QLoRA on Axolotl over 3 epochs.
• Prompt Format: Utilizes the ChatML prompt format, with a specific system prompt example provided for uncensored and unbiased AI assistance.

Intended Use Cases

• Coding: Excels particularly in coding-related applications due to its specialized training data.
• General Chat: While focused on coding, future plans for Dolphin 3.0 include enhancements for general chat, structured output, agent cases, and role-playing.

This model was developed with sponsorship from MassedCompute and acknowledges contributions from various open-source community members and dataset authors.