dphn/dolphin-2.2.1-mistral-7b
dphn/dolphin-2.2.1-mistral-7b is a 7 billion parameter language model developed by Eric Hartford, based on MistralAI's Mistral-7B-v0.1 architecture. This model is fine-tuned for enhanced conversation, empathy, and multi-turn dialogue, incorporating elements from Microsoft's Orca and Samantha datasets. It is designed to be highly compliant and uncensored, making it suitable for diverse applications where custom alignment layers can be implemented.
Loading preview...
Dolphin 2.2.1 Mistral 7B Overview
dphn/dolphin-2.2.1-mistral-7b is a 7 billion parameter language model developed by Eric Hartford, building upon the MistralAI Mistral-7B-v0.1 base. This iteration, Dolphin 2.2.1, specifically addresses overfit training issues from previous versions, aiming for more natural responses without unrequested Chain-of-Thought (CoT) reasoning and improved compliance.
Key Capabilities & Features
- Enhanced Conversation and Empathy: New in version 2.2, the model has been infused with curated Samantha DNA and additional training for long, multi-turn conversations, allowing it to offer personal advice and demonstrate empathy.
- Uncensored and Highly Compliant: The dataset has been filtered to remove alignment and bias, resulting in a model that is highly compliant to user requests, including potentially unethical ones. Users are advised to implement their own alignment layers for service deployment.
- Diverse Dataset Integration: Training utilized a modified Dolphin dataset (an open-source Orca implementation), enhanced with Jon Durbin's Airoboros dataset for increased creativity, and a curated subset of WizardLM and Samantha for conversational abilities.
- Commercial Use: Based on the Apache-2.0 licensed MistralAI model, Dolphin 2.2.1 is suitable for both commercial and non-commercial applications.
- ChatML Prompt Format: The model uses the ChatML prompt format for consistent interaction.
Training Details
The model underwent 4 epochs of training over 48 hours on 4x A100 GPUs. The training process involved specific modifications to the dataset for uncensoring, deduplication, cleaning, and quality improvement.