HeAAAAA/Crab
Crab is an 8 billion parameter configurable role-playing (RP) large language model developed by HeAAAAA, built upon the Llama-3.1-8B architecture. It is designed for dynamic role configuration in dialogue generation, moving beyond models with preset roles. Crab excels in generating role-based behaviors and emotions, supported by a comprehensive RP training dataset and a novel evaluation benchmark, RoleRM. This model is optimized for flexible and adaptable role-playing scenarios.
Loading preview...
Overview
Crab is a novel Configurable Role-Playing (RP) LLM developed by HeAAAAA, featuring an 8 billion parameter architecture based on Llama-3.1-8B. Unlike traditional RP models that rely on several preset roles, Crab enables dynamic configuration of desired roles, significantly enhancing flexibility and adaptability in dialogue generation. It was trained on the largest curated RP training dataset, which includes detailed role overviews, character profiles, conversation scenarios, and tagged topics.
Key Capabilities & Innovations
- Dynamic Role Configuration: Generates dialogues dynamically from configurations rather than memorizing specific roles, allowing for a diverse range of roles with minimal dialogue per role.
- Comprehensive RP Dataset: Utilizes a large training dataset capturing a broad spectrum of role-based behaviors, emotions, and interactions.
- Novel Evaluation Benchmark (RoleRM): Introduces a new benchmark with an evaluation standard, a manually annotated test dataset, and a reward model (RoleRM) designed to automatically assess specific aspects of RP, aligning with human perception. RoleRM significantly outperforms ChatGPT and other methods in fine-grained RP evaluations.
- Superior Performance: Experiments demonstrate that Crab-powered models, particularly Llama-3.1-8B-Crab, achieve superior performance across various fine-grained aspects of role-playing, including Language Fluency, Role Language, Role Knowledge, and Emotional Expression, outperforming other Llama variants, GPT models, and specialized RP-LLMs like Pygmalion-2-7B.
Datasets & Resources
HeAAAAA has released four related datasets:
- Crab role-playing train set: For fine-tuning RP-LLMs.
- Crab role-playing evaluation benchmark: For evaluating RP-LLMs.
- Manually annotated role-playing evaluation dataset: For training RP evaluators.
- Crab Human preference dataset: For training RP-LLMs via reinforcement learning.
When to Use
Crab is ideal for developers and researchers focused on advanced, flexible, and highly configurable role-playing applications where dynamic character interaction and nuanced emotional expression are critical. Its robust evaluation framework also makes it suitable for research into RP model performance.