Dolphin 2.8 Mistral 7b v0.2 Overview

Dolphin 2.8 Mistral 7b v0.2 is a 7 billion parameter language model developed by Eric Hartford and Cognitive Computations. It is built upon the Mistral-7b-v0.2 base model, which features a 32k context length, and was fine-tuned using 16k sequence lengths. The model was trained over three days on 10x L40S GPUs.

Key Capabilities

• Instruction Following: Excels at understanding and executing diverse instructions.
• Conversational AI: Capable of engaging in varied conversational tasks.
• Coding Skills: Possesses abilities in code generation and related tasks.
• Uncensored Nature: The model is uncensored and highly compliant to requests, including potentially unethical ones, as its dataset was filtered to remove alignment and bias. Users are advised to implement their own alignment layers.

Performance Metrics

Evaluations show competitive performance across several benchmarks:

• ARC-Challenge: 0.592 acc
• GSM8K: 0.478 exact match
• HellaSwag: 0.638 acc
• MMLU: 0.612 acc
• TruthfulQA_MC2: 0.518 acc
• Winogrande: 0.797 acc

Licensing

Dolphin 2.8 is licensed under Apache 2.0, permitting commercial use. It was trained on data generated from GPT-4, among other models.