Model Overview
Sao10K/L3-8B-Stheno-v3.2 is an 8 billion parameter model, fine-tuned by Sao10K from a Llama-3 base. This version, Stheno-v3.2-Zeta, represents the sixth iteration of development, focusing on refining its generative capabilities across various tasks. The training involved approximately 24 hours on a single H100 SXM GPU, with hyperparameter tuning to achieve lower loss levels.
Key Enhancements & Capabilities
- Balanced Content Generation: Improved handling of both SFW (Safe For Work) and NSFW (Not Safe For Work) content, achieving a more balanced output compared to previous versions.
- Enhanced Storywriting & Narration: Demonstrates better performance in generating coherent and engaging stories and narrative passages.
- Improved Assistant-Style Tasks: More effective at handling instruction-based and assistant-type queries.
- Multi-Turn Coherency: Features better consistency and reduced issues in multi-turn conversations, maintaining context more effectively.
- Prompt Adherence: Shows stronger adherence to given prompts and instructions, leading to more predictable and controlled outputs.
Training Data & Methodology
This version incorporated a mix of SFW and NSFW storywriting data, including contributions from Gryphe's Opus-WritingPrompts dataset. It also included more instruct/assistant-style data and underwent further cleaning of roleplaying samples to remove low-quality entries. While slightly less creative than its predecessor, this is considered a worthwhile trade-off for improved control and coherency.
Recommended Usage
Users are advised to use specific sampling parameters for optimal performance:
- Temperature: 1.12-1.22
- Min-P: 0.075
- Top-K: 50
- Repetition Penalty: 1.1
The model utilizes the Llama-3-Instruct prompting template and includes recommended stopping strings for effective interaction.