lemonilia/LimaRP-Mistral-7B-v0.1

LimaRP-Mistral-7B-v0.1: Longform Roleplay Model

This model, developed by lemonilia, is a 7 billion parameter variant of Mistral-7B-v0.1, specifically fine-tuned for longform, novel-style 1-on-1 roleplaying chat. It aims to replicate the immersive experience of Internet forum-based roleplay, distinguishing itself from short-form chat models.

Key Capabilities & Features

• Specialized Roleplay: Optimized for detailed, narrative-driven roleplay conversations, focusing on persona and scenario data.
• Extended Alpaca Format: Uses a modified Alpaca prompt structure, including ### Instruction:, ### Input:, and ### Response: for multi-turn interactions.
• Dynamic Message Length Control: A unique feature allowing users to append length modifiers (e.g., (length = medium)) to the response instruction, influencing the model's output length. Available lengths range from micro to unlimited, with medium recommended as a starting point.
• No Instruction Tuning: The model is trained exclusively on manually selected and edited roleplay conversations, without general instruction tuning.

Training Details

The model was trained using Axolotl on 4x NVidia A40 GPUs, with approximately 2000 training samples up to 9k tokens in length. The training involved one epoch, with data repeated in slightly different formats, and a second epoch utilizing a differently arranged system instruction. Key hyperparameters included a learning rate of 0.0003 and a sequence length of 8750.

Recommended Use Cases

• Immersive Roleplaying: Ideal for users seeking detailed, character-driven narrative interactions.
• Customizable Response Lengths: Beneficial for scenarios where precise control over the length of AI-generated responses is desired.
• Forum-Style RP: Best suited for replicating the experience of traditional, longform roleplay found on internet forums.