Model Overview

zerofata/L3.3-GeneticLemonade-Unleashed-v3-70B is an experimental 70 billion parameter model based on the Llama-3 architecture, fine-tuned by zerofata. It is a Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) QLora finetune, building upon the zerofata/GeneticLemonade-Unleashed-70B base.

Key Capabilities

• Character-Driven Roleplay (RP/ERP): Specifically designed to excel in character-driven and erotic roleplay scenarios.
• Narrative-Heavy Responses: Generates longer, detailed responses that focus on storytelling and character development.
• Proactive Character Portrayal: Aims for characters to be accurately and proactively represented within the narrative.
• High Temperature Tolerance: Noted to support higher inference temperatures (0.9 - 1.2) than typically recommended for other Llama 3 models, allowing for more creative and varied outputs.

Training Process

The model underwent a two-stage training process:

• SFT Phase: Initial Supervised Fine-Tuning with a synthetic dataset of 2.9 million tokens, comprising approximately 750 conversations, primarily RP data with some instruct/assistant and creative writing.
• DPO Phase: Subsequent Direct Preference Optimization using around 1100 high-quality chosen examples from the SFT dataset, with rejected samples generated by a Llama 3.3 finetune known for poor instruction following.

Recommended Use Cases

• Interactive Storytelling: Ideal for applications requiring deep, character-focused narratives.
• Roleplaying Bots: Excels in scenarios where accurate and proactive character responses are crucial.

This model is not optimized for general creative writing or adventure stories, but rather for specific character-centric interactions.