Model Overview
MuXodious/Harbinger-24B-absolute-heresy is a 24 billion parameter language model fine-tuned from Mistral Small 3.1 Instruct. It was developed by MuXodious in collaboration with Gryphe Padar, with a focus on creating immersive text adventures and roleplay experiences. The model underwent a two-stage training process, including Supervised Fine-Tuning (SFT) on multi-turn datasets and Direct Preference Optimization (DPO) using reward model user preference data.
Key Capabilities
- Enhanced Narrative Coherence: Specifically trained to improve storytelling flow and consistent character behaviors over long sequences.
- Instruction Following: Strengthened ability to adhere to user instructions, crucial for interactive narratives.
- Reduced Repetition: DPO techniques were applied to minimize clichés, repetitive patterns, and common AI artifacts in generated text.
- Second-Person Present Tense: Primarily trained on data in this style, making it highly effective for "you are" narratives.
Unique Characteristics
This model has undergone a process referred to as "Heretication" using P-E-W's Heretic engine, resulting in an "Absolute Heresy" classification with 4/100 refusals and a KL Divergence of 0.0210. While this classification is arbitrary and inspired by Warhammer 40K, it indicates a significant modification from its base model. The model is sensitive to higher temperatures, with recommended inference settings including temperature: 0.8, repetition_penalty: 1.05, and min_p: 0.025.
Ideal Use Cases
- Immersive Text Adventures: Designed for games and stories where consequences and decisions are central.
- Roleplay Scenarios: Excels at maintaining consistent character behavior and narrative flow in roleplaying contexts.
- Creative Writing: Suitable for generating polished, engaging narratives, particularly in the second-person present tense.