Aura-8B: A Dedicated Roleplaying Model

Aura-8B, developed by Aura Industries with contributions from Anthracite Org, is an 8 billion parameter instruction-tuned model built upon the arcee-ai/Llama-3.1-SuperNova-Lite base. Its primary distinction lies in its specialized optimization for roleplaying scenarios, achieved through extensive fine-tuning on hundreds of millions of tokens of instruction and roleplaying data.

Key Capabilities & Features

• Dedicated Roleplaying: Engineered specifically to excel in interactive and narrative roleplaying applications.
• Unique Output Style: Incorporates a Kahneman-Tversky Optimization (KTO) as a Low Rank Adapter, contributing to a distinct conversational and narrative style.
• Extended Context Window: Supports a maximum context length of 8,192+ tokens, allowing for more complex and sustained interactions.
• Llama 3 Prompt Format: Utilizes the Llama 3 prompt format for chat completions.
• Quantizations Available: Offers various quantizations including Static GGUF, Imatrix GGUF, and EXL2 for flexible deployment.

Training & Performance

The model underwent a two-stage training process: an initial Supervised Fine-Tuning (SFT) phase followed by a Kahneman-Tversky Optimization (KTO) phase. The SFT phase leveraged diverse datasets covering roleplaying, cybersecurity, medical instruction, math, and creative writing. The KTO phase used the anthracite-core/full-opus-chosen-hermes-rejected-kto-v1 dataset to refine its output style. On the Open LLM Leaderboard, Aura-8B achieved an average score of 27.34, with notable scores in IFEval (72.05) and MMLU-PRO (31.93).

Ideal Use Cases

Aura-8B is particularly well-suited for applications requiring:

• Interactive Storytelling: Generating dynamic and engaging narratives.
• Character Simulation: Creating realistic and consistent character personas for virtual companions or game NPCs.
• Creative Writing Assistance: Aiding in the development of fictional scenarios and dialogues.
• Personalized Conversational Agents: Building chatbots with distinct personalities and role-specific interactions.