Sao10K/32B-Qwen2.5-Kunou-v1

Sao10K/32B-Qwen2.5-Kunou-v1: A Generalist Roleplay Model

This model, developed by Sao10K, is a 32.8 billion parameter variant built upon the Qwen2.5 architecture. It is positioned as a general-purpose model with a particular emphasis on roleplay scenarios, serving as a spiritual successor to the L3-70B-Euryale-v2.2 series but based on a different foundational architecture. The development involved a significantly cleaned and improved dataset compared to its predecessors.

Key Capabilities & Features

• Generalist Performance: Designed for broad applicability across various language tasks.
• Enhanced Roleplay: Specifically fine-tuned with a focus on creative and interactive roleplay, utilizing a refined dataset.
• Large Context Window: Supports a substantial context length of 131072 tokens, enabling longer and more coherent interactions.
• Optimized Training: Built using Axolotl with QLoRA, featuring specific optimizations like liger_rope, liger_rms_norm, and liger_glu_activation for improved efficiency and performance.

Recommended Usage

For optimal performance, the model recommends using the ChatML prompt format with a temperature of 1.1 and min_p of 0.1. The developer emphasizes that the system prompt plays a crucial role in guiding the model's output. Future plans include further refining the datasets to enhance quality, particularly for secondary chats and creative-related domains.