0xA50C1A1/Llama-3.3-8B-Darkmere
Llama-3.3-8B-Darkmere by 0xA50C1A1 is an 8 billion parameter Llama 3.3 Instruct fine-tune with an 8192 token context length. It is specifically optimized for roleplay and creative writing tasks, particularly in narrative-dense genres like horror and NSFW scenarios. This model is also designed for merging with other Llama 3.1 8B fine-tunes.
Loading preview...
Overview
Llama-3.3-8B-Darkmere is an 8 billion parameter language model developed by 0xA50C1A1, fine-tuned from the Llama 3.3 Instruct architecture. It features an 8192 token context length and was trained using a full fine-tuning method over 3 epochs with a learning rate of 5e-6.
Key Capabilities
- Specialized for Creative Writing: This model is specifically fine-tuned for generating creative text, with a particular emphasis on roleplay scenarios.
- Narrative Density: Training on the refined Darkmere dataset, which includes human-written horror and NSFW content, enhances its ability to produce rich, detailed narratives.
- Merging Compatibility: It is designed to be compatible for merging with other Llama 3.1 8B fine-tunes, offering flexibility for further customization.
Training Details
The model was trained on the Darkmere dataset, which has been refined to increase narrative density and incorporate specific genres. The training process involved full fine-tuning, leveraging the Llama 3.3 8B weights provided by allura-forge and shb777, and utilizing AMD Instinct™ MI300X GPUs. The Heretic tool by p-e-w was instrumental in preparing the training data by assisting with censorship removal.