MuXodious/Harbinger-24B-absolute-heresy
MuXodious/Harbinger-24B-absolute-heresy is a 24 billion parameter fine-tuned causal language model based on Mistral Small 3.1 Instruct, developed by MuXodious in collaboration with Gryphe Padar. It is specifically optimized for immersive text adventures and roleplay, enhancing instruction following, mid-sequence continuation, and narrative coherence over long sequences. This model utilizes Direct Preference Optimization (DPO) to produce polished outputs with fewer clichés and repetitive patterns, and has a context length of 32768 tokens.
Loading preview...
Model Overview
MuXodious/Harbinger-24B-absolute-heresy is a 24 billion parameter language model fine-tuned from Mistral Small 3.1 Instruct. It was developed by MuXodious in collaboration with Gryphe Padar, with a focus on creating immersive text adventures and roleplay experiences. The model underwent a two-stage training process, including Supervised Fine-Tuning (SFT) on multi-turn datasets and Direct Preference Optimization (DPO) using reward model user preference data.
Key Capabilities
- Enhanced Narrative Coherence: Specifically trained to improve storytelling flow and consistent character behaviors over long sequences.
- Instruction Following: Strengthened ability to adhere to user instructions, crucial for interactive narratives.
- Reduced Repetition: DPO techniques were applied to minimize clichés, repetitive patterns, and common AI artifacts in generated text.
- Second-Person Present Tense: Primarily trained on data in this style, making it highly effective for "you are" narratives.
Unique Characteristics
This model has undergone a process referred to as "Heretication" using P-E-W's Heretic engine, resulting in an "Absolute Heresy" classification with 4/100 refusals and a KL Divergence of 0.0210. While this classification is arbitrary and inspired by Warhammer 40K, it indicates a significant modification from its base model. The model is sensitive to higher temperatures, with recommended inference settings including temperature: 0.8, repetition_penalty: 1.05, and min_p: 0.025.
Ideal Use Cases
- Immersive Text Adventures: Designed for games and stories where consequences and decisions are central.
- Roleplay Scenarios: Excels at maintaining consistent character behavior and narrative flow in roleplaying contexts.
- Creative Writing: Suitable for generating polished, engaging narratives, particularly in the second-person present tense.