ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.1

Model Overview

ArliAI-RPMax-12B-v1.1 is a 12 billion parameter model from ArliAI's RPMax series, built upon the Mistral Nemo 12B Instruct 2407 base. This variant is highlighted as a particularly successful RPMax model, leveraging Mistral's inherently uncensored nature to enhance its creative capabilities.

Key Capabilities

• Creative Writing & Roleplay: Specifically trained on diverse, curated, and deduplicated creative writing and roleplay datasets.
• Non-Repetitive Generation: Designed to avoid repetition in characters or situations, ensuring varied and dynamic outputs.
• Adaptable Personalities: Capable of understanding and appropriately acting as various characters without developing a fixed persona.
• High Versatility: Aims to prevent "in-bred" feeling often found in other roleplay models, offering a fresh and distinct style.

Training Details

The model was trained for approximately 2 days on 2x3090Ti GPUs, utilizing a sequence length of 8192. It underwent 1 epoch of training to minimize repetition sickness, employing QLORA with 64-rank and 128-alpha, resulting in roughly 2% trainable weights. A learning rate of 0.00001 and a low gradient accumulation of 32 were used for optimized learning.

Benchmarks

Evaluated on the Open LLM Leaderboard, the model achieved an average score of 20.64. Notable scores include 53.49 on IFEval (0-Shot) and 26.49 on MMLU-PRO (5-shot).

Good For

• Developers seeking a highly creative and adaptable model for roleplaying applications.
• Generating diverse and non-repetitive narrative content.
• Use cases requiring dynamic character interaction and story progression.