nbeerbower/Qwen3-Gutenberg-Encore-14B
Qwen3-Gutenberg-Encore-14B is a 14 billion parameter language model developed by nbeerbower, based on the Qwen3 architecture. This model is specifically fine-tuned using ORPO on a collection of DPO datasets derived from Gutenberg and synthetic fiction, focusing on enhancing its capabilities for creative writing and narrative generation. It excels at producing high-quality, coherent long-form text, making it suitable for applications requiring advanced storytelling or literary content creation.
Loading preview...
Qwen3-Gutenberg-Encore-14B Overview
Qwen3-Gutenberg-Encore-14B is a 14 billion parameter language model developed by nbeerbower, built upon the base nbeerbower/Xiaolong-Qwen3-14B model. Its primary distinction lies in its specialized fine-tuning process, which leverages a technique called ORPO (Optimized Reward-Prompt Optimization) to enhance its performance in specific domains.
Key Capabilities
- Enhanced Narrative Generation: The model has been fine-tuned on a diverse set of DPO (Direct Preference Optimization) datasets, including:
- jondurbin/gutenberg-dpo-v0.1
- nbeerbower/gutenberg2-dpo
- nbeerbower/gutenberg-moderne-dpo
- nbeerbower/synthetic-fiction-dpo
- nbeerbower/Arkhaios-DPO
- nbeerbower/Purpura-DPO
- nbeerbower/Schule-DPO
This extensive training on literary and synthetic fiction datasets makes it particularly adept at generating creative and coherent long-form text.
- ORPO Fine-tuning: The model was tuned using the ORPO method, a technique designed to align model outputs with human preferences more effectively. This process involved 3 epochs of training on a single RTX A6000 GPU.
- QLoRA Configuration: It utilizes QLoRA with specific parameters (r=64, lora_alpha=128, lora_dropout=0.05) for efficient fine-tuning, targeting key attention and feed-forward projection layers.
Good for
- Creative Writing: Generating stories, novels, scripts, or other forms of long-form narrative content.
- Literary Analysis & Augmentation: Assisting with tasks related to existing literary works, such as expanding on themes or creating alternative endings.
- Content Generation: Producing high-quality, engaging text for various applications where a strong narrative voice is desired.