ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.3

TEXT GENERATIONConcurrent Unit Cost:1Model Size:12BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Dec 5, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

ArliAI's Mistral-Nemo-12B-ArliAI-RPMax-v1.3 is a 12 billion parameter language model, part of the RPMax series, fine-tuned for creative writing and role-playing tasks. It features a 32,768 token context length and is specifically designed to reduce cross-context repetition and enhance creative output. This model aims to provide varied and non-repetitive responses by leveraging a uniquely curated and deduplicated dataset.

Loading preview...

ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.3 Overview

This model is a 12 billion parameter variant from ArliAI's RPMax series, specifically fine-tuned from the Mistral-Nemo-12B-Instruct base model. It is designed to excel in creative writing and role-playing (RP) scenarios by significantly reducing repetition and fostering higher creativity in its outputs. The v1.3 update incorporates improvements from updated software and configurations, including fixes to the transformers library's gradient checkpointing bug, leading to better learning.

Key Capabilities & Differentiators

  • Reduced Repetition: Focuses on eliminating "cross-context repetition," where models repeat phrases or tropes across different conversations, ensuring more unique and varied responses.
  • Enhanced Creativity: Aims for diverse output, not just pleasant prose, by preventing the model from over-fitting to specific styles or tropes.
  • Unique Training Methodology: Utilizes an unconventional training approach with a single epoch, low gradient accumulation, and a higher learning rate. This method, combined with rank-stabilized low-rank adaptation (RS-QLORA+), encourages the model to learn from each example without reinforcing specific patterns.
  • Curated Dataset: Trained on a diverse, deduplicated dataset of creative writing and RP examples, carefully filtered to remove synthetic generations and ensure no repeated characters or situations.
  • Context Length: Supports a substantial context length of 32,768 tokens.

Ideal Use Cases

  • Creative Writing: Generating diverse and non-repetitive narratives.
  • Role-Playing (RP): Engaging in dynamic and unpredictable RP scenarios where character consistency and novel interactions are desired.
  • Applications Requiring Varied Output: Any application where avoiding predictable or generic responses is crucial.

Finetunes
2 models