MN-12b-RP-Ink: A Roleplay-Focused Mistral Fine-tune

MN-12b-RP-Ink is a 12 billion parameter model developed by allura-org, built upon the Mistral Nemo Instruct architecture. It is a LoRA fine-tune specifically designed for roleplay (RP) generation, distinguishing itself through its unique training methodology and dataset.

Key Characteristics & Training

• Base Model: Mistral Nemo Instruct.
• Fine-tuning: Utilizes a LoRA (Low-Rank Adaptation) approach with specific hyperparameters:
  • Rank: 16
  • Alpha: 32
  • Dropout: 0.25 (inspired by Slush).
• Training Regimen: Trained for 2 epochs with a learning rate of 6e-5, using a Cosine LR Scheduler and Paged AdamW 8bit optimizer, with an effective batch size of 12.
• Dataset: The model was trained on a highly unconventional and diverse dataset, described as a "worst mix of data" by its creators, which contributes to its distinct roleplay generation capabilities.

Recommended Usage

• Chat Template: Mistral v3-Tekken is the recommended chat template for optimal performance.
• Sampler Settings: Users are encouraged to experiment, but suggested sampler settings include:
  • Temperature 1.25 / MinP 0.1
  • Temperature 1.03 / TopK 200 / MinP 0.05 / TopA 0.2

Why Choose MN-12b-RP-Ink?

This model is particularly suited for applications requiring highly creative, engaging, and character-driven narrative generation. Its specialized fine-tuning makes it a strong candidate for roleplaying scenarios, interactive storytelling, and other creative text generation tasks where a distinct and dynamic voice is desired.