MemGPT/dolphin-2.2-yi-34b-200k
MemGPT/dolphin-2.2-yi-34b-200k is a 34 billion parameter language model developed by Eric Hartford, based on the Yi architecture. It features a 32K context length, fine-tuned for multi-turn conversation and empathy, incorporating curated Samantha and WizardLM DNA. This model is uncensored and highly compliant, making it suitable for diverse conversational applications where custom alignment layers can be implemented.
Loading preview...
Dolphin 2.2 Yi-34b-200k Overview
This model, developed by Eric Hartford and sponsored by Convai, is a 34 billion parameter language model built upon the Yi architecture. While the base Yi model supports a 200k context, Dolphin-2.2-Yi-34b-200k was fine-tuned with a 16k context length. A key focus of this 2.2 release is enhanced conversation and empathy, achieved through an infusion of curated Samantha and WizardLM datasets, specifically optimized for long, multi-turn interactions.
Key Capabilities
- Enhanced Conversational Ability: Trained for extended, multi-turn dialogues.
- Empathy and Personal Advice: Designed to offer personal advice and respond with empathy.
- Uncensored and Compliant: The dataset was filtered to remove alignment and bias, resulting in a highly compliant model that will follow requests, including potentially unethical ones. Users are advised to implement their own alignment layers.
- Creative Text Generation: Incorporates Jon Durbin's Airoboros dataset to boost creativity.
Training Details
The model was trained for 3 epochs over 3 days using qLoRA and Axolotl on 4x A100 GPUs. It utilizes the ChatML prompt format for interaction.