dphn/dolphin-2_2-yi-34b
Dolphin-2.2-Yi-34b is a 34 billion parameter instruction-tuned causal language model developed by dphn, based on the Yi architecture. Fine-tuned with a 16k context window, this model emphasizes conversation and empathy, incorporating curated Samantha and WizardLM data for enhanced multi-turn dialogue. It is designed to be highly compliant and uncensored, making it suitable for applications where custom alignment layers are implemented.
Loading preview...
Model Overview
Dolphin-2.2-Yi-34b is a 34 billion parameter instruction-tuned language model built upon the Yi architecture, specifically using the chargoddard/Yi-34B-Llama base. Developed by dphn and sponsored by a16z, this iteration, Dolphin 2.2, introduces significant enhancements in conversational ability and empathy. It was trained with a 16k context window over three epochs using qLoRA and Axolotl on 4x A100 GPUs.
Key Capabilities
- Enhanced Conversation and Empathy: Infused with curated Samantha and WizardLM data, the model excels at multi-turn conversations and can provide personal advice, demonstrating an understanding of user feelings.
- Uncensored and Compliant: The dataset was filtered to remove alignment and bias, resulting in a highly compliant model that will follow requests, including potentially unethical ones. Users are advised to implement their own alignment layers.
- Dataset Diversity: The training dataset is an open-source implementation of Microsoft's Orca, modified for uncensoring, deduping, and quality. It also integrates Jon Durbin's Airoboros dataset for increased creativity.
- ChatML Prompt Format: The model utilizes the ChatML prompt format, ensuring consistent and structured interaction.
Use Cases
- Applications requiring highly compliant and uncensored responses, where developers can implement custom safety measures.
- Scenarios demanding nuanced multi-turn conversations and empathetic interactions.
- Creative content generation and complex problem-solving due to the inclusion of the Airoboros dataset.