Overview

crestf411's L3.1-8B-Slush-v1.1 is an 8 billion parameter model built upon the Llama 3.1 architecture, designed to address the base model's limitations in creativity and imagination. It employs a unique two-stage training methodology:

• Stage 1 (Pretraining Continuation): Focuses on boosting the model's creativity and writing abilities through high LoRA dropout, then merging this into the instruction-tuned base.
• Stage 2 (Fine-tuning): Further refines roleplaying capabilities and mitigates any potential degradation from the initial merge.

This model incorporates a custom merge using MergeKit with the TIES method, targeting the meta-llama/Llama-3.1-8B as its base. Training parameters were adjusted in v1.1 based on feedback, including specific LoRA configurations (rank 64, alpha 128 for stage 1; rank 32, alpha 64 for stage 2) and the use of LoRA+.

Key Capabilities

• Enhanced Creativity: Aims to improve imaginative text generation.
• Improved Writing: Designed for better overall writing quality.
• Strong Roleplaying: Specifically fine-tuned to excel in interactive roleplay scenarios, following the Silly Tavern preset.

Good For

• Applications requiring creative text generation.
• Interactive storytelling and roleplaying.
• Use cases where imagination and nuanced responses are critical.