Dolphin 2.0 Mistral 7B Overview
sabatico/dolphin-2.0-mistral-7b is an uncensored 7 billion parameter language model built on the MistralAI architecture, developed by Eric Hartford with sponsorship from a16z. This model is designed for high compliance, meaning it will follow user instructions closely, even potentially unethical ones, and users are advised to implement their own alignment layers for responsible deployment.
Key Capabilities & Training
- Uncensored Nature: The dataset was filtered to remove alignment and bias, resulting in a highly compliant model.
- Dataset: Based on Dolphin, an open-source implementation of Microsoft's Orca, with modifications for uncensoring, deduping, cleaning, and quality. It also integrates Jon Durbin's Airoboros dataset to boost creativity.
- Training: The model underwent 10 epochs of training over 48 hours on 4x A100 GPUs.
- Prompt Format: Utilizes the ChatML prompt format, starting with
<|im_start|>system and <|im_start|>user tags.
Use Cases & Considerations
This model is suitable for both commercial and non-commercial applications due to its MistralAI base. Its uncensored nature makes it highly adaptable to specific user requirements, but also places the responsibility on the user to implement necessary safety and alignment measures. It excels in scenarios requiring a highly compliant model that can be tailored with custom guardrails.