Hugging Face Overview
Model Overview
zerofata/L3.3-GeneticLemonade-Unleashed-v3-70B is an experimental 70 billion parameter model based on the Llama-3 architecture, fine-tuned by zerofata. It is a Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) QLora finetune, building upon the zerofata/GeneticLemonade-Unleashed-70B base.
Key Capabilities
- Character-Driven Roleplay (RP/ERP): Specifically designed to excel in character-driven and erotic roleplay scenarios.
- Narrative-Heavy Responses: Generates longer, detailed responses that focus on storytelling and character development.
- Proactive Character Portrayal: Aims for characters to be accurately and proactively represented within the narrative.
- High Temperature Tolerance: Noted to support higher inference temperatures (0.9 - 1.2) than typically recommended for other Llama 3 models, allowing for more creative and varied outputs.
Training Process
The model underwent a two-stage training process:
- SFT Phase: Initial Supervised Fine-Tuning with a synthetic dataset of 2.9 million tokens, comprising approximately 750 conversations, primarily RP data with some instruct/assistant and creative writing.
- DPO Phase: Subsequent Direct Preference Optimization using around 1100 high-quality chosen examples from the SFT dataset, with rejected samples generated by a Llama 3.3 finetune known for poor instruction following.
Recommended Use Cases
- Interactive Storytelling: Ideal for applications requiring deep, character-focused narratives.
- Roleplaying Bots: Excels in scenarios where accurate and proactive character responses are crucial.
This model is not optimized for general creative writing or adventure stories, but rather for specific character-centric interactions.