Model Overview
nbeerbower/Mistral-Nemo-Gutenberg-Encore-12B is a 12 billion parameter language model derived from the Mistral-Nemo-Instruct-2407 architecture. It has been specifically fine-tuned using the ORPO (Optimized Reward Policy Optimization) method to excel in creative writing tasks, particularly fiction generation.
Key Capabilities
- Enhanced Narrative Quality: The model demonstrates improved story arcs, pacing, and coherent plot development compared to its base model.
- Refined Prose Style: It generates text with more evocative imagery and thematic depth, moving beyond generic phrasing.
- Thematic Exploration: Excels at handling complex themes like sacrifice, entropy, and memory, integrating them effectively into narratives.
- Prompt Adherence: Highly capable of following detailed and imaginative writing prompts, incorporating specific elements and stylistic instructions.
Training Details
The model was ORPO tuned over 3 epochs using a single RTX A6000 GPU. The training leveraged several DPO datasets curated by nbeerbower, including gutenberg-dpo-v0.1, gutenberg2-dpo, gutenberg-moderne-dpo, synthetic-fiction-dpo, Arkhaios-DPO, Purpura-DPO, and Schule-DPO. This specialized dataset focus aims to imbue the model with a strong understanding of literary structures and styles.
Use Cases
This model is particularly well-suited for:
- Generating creative fiction, short stories, and narrative content.
- Developing imaginative scenarios and detailed world-building.
- Assisting writers with plot development and character interactions.
- Applications requiring nuanced and thematically rich text generation.