nbeerbower/Gemma2-Gutenberg-Doppel-9B
nbeerbower/Gemma2-Gutenberg-Doppel-9B is a 9 billion parameter language model based on the Gemma 2 architecture, fine-tuned using the ORPO method. It specializes in generating text aligned with the style and content found in the Gutenberg Project, making it suitable for literary analysis, historical text generation, and creative writing tasks. The model leverages a 16384 token context length for processing longer inputs and maintaining coherence.
Loading preview...
Model Overview
nbeerbower/Gemma2-Gutenberg-Doppel-9B is a 9 billion parameter language model derived from the UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3 base model. It has been fine-tuned using the ORPO (Optimized Reward-Prompt Optimization) method over three epochs, utilizing two A40 GPUs. The training incorporated two specific datasets: jondurbin/gutenberg-dpo-v0.1 and nbeerbower/gutenberg2-dpo, which are focused on content from the Gutenberg Project.
Key Capabilities
- Gutenberg-aligned Text Generation: Excels at producing text that reflects the style, vocabulary, and themes prevalent in classic literature and historical documents found in the Gutenberg Project.
- ORPO Fine-tuning: Benefits from the ORPO method, which aims to improve alignment and response quality.
- Extended Context Window: Supports a context length of 16384 tokens, allowing for the processing and generation of longer, more coherent passages.
Performance Metrics
Evaluated on the Open LLM Leaderboard, the model achieved an average score of 29.82. Notable individual metric scores include:
- IFEval (0-Shot): 71.71
- BBH (3-Shot): 41.08
- MMLU-PRO (5-shot): 34.75
Ideal Use Cases
- Literary Analysis: Generating summaries, character analyses, or stylistic imitations of classic texts.
- Historical Content Creation: Producing narratives, dialogues, or descriptions in a historical context.
- Creative Writing: Assisting with fiction writing, particularly for genres that benefit from a more classical or formal tone.
- Educational Tools: Developing resources for studying literature or historical periods.