Perexiguus-0.6B: A Compact Roleplaying LLM
hamzah0asadullah's Perexiguus-0.6B is a 0.6 billion parameter model, fine-tuned from Qwen/Qwen3-0.6B, specifically designed for roleplaying conversations. It stands out as one of the smallest models capable of generating usable roleplay interactions, trained on approximately 0.05 billion tokens of diverse, roleplay-concentrated data.
Key Capabilities & Features
- Optimized for Roleplay: Fine-tuned on a dataset of roleplay-focused conversations, enabling it to generate engaging and consistent character interactions.
- Compact Size: With 0.6 billion parameters, it is highly efficient and suitable for environments with limited computational resources.
- Flexible Prompting: Supports various prompt formats, including detailed system prompts and initial assistant messages, similar to Character.AI's style.
- Architectural Foundation: Retains the causal language model architecture of its base, Qwen3-0.6B, featuring 28 layers and Grouped-Query Attention (GQA) with 16 query heads and 8 Key-Value heads.
- Native Context Length: Offers a substantial native context length of 32,768 tokens.
Usage Recommendations
- Sampling Parameters: Recommended settings include Top-K between 5-15, Top-P at 0.95, Temperature between 0.6-0.8, and a Repetition Penalty of 1.15.
- System Prompts: Performance is significantly enhanced with clear and detailed system prompts. Avoid direct addressing of the model with "you" when describing characters.
- Roleplay Specifics: For roleplaying, it is recommended to add
\n/no_think to the first message to prevent degraded reasoning loops. - Quantization: Due to its small size, using quantizations below INT8/FP8 is not recommended to maintain coherent responses.