Overview

nbeerbower/llama3.1-gutenberg-8B is an 8 billion parameter language model derived from VAGOsolutions/Llama-3.1-SauerkrautLM-8b-Instruct. This model has undergone additional fine-tuning to enhance its capabilities, particularly in areas related to literary text.

Key Capabilities

• Literary Text Generation: Optimized for generating text with stylistic elements found in classic literature.
• Creative Writing: Excels at tasks requiring imaginative and nuanced textual output.
• Text Understanding: Improved comprehension of literary contexts and themes.

Training Details

The model was fine-tuned using 2x RTX 4060 GPUs over 3 epochs. The training utilized the jondurbin/gutenberg-dpo-v0.1 dataset, which focuses on literary content, to specialize its output. The fine-tuning method was based on techniques described in "Fine-tune Llama 3 with ORPO" by mlabonne.

Good for

• Generating stories, poems, or prose in various literary styles.
• Assisting with creative writing projects and overcoming writer's block.
• Developing applications that require text generation with a sophisticated or classic literary tone.