RicardoEstep/RPBizkit-v11-12B-Base
RicardoEstep/RPBizkit-v11-12B-Base is a 12.2 billion parameter experimental Mistral Nemo mix model, created by RicardoEstep, designed for roleplay-oriented applications. This model is a merge of several 'RP Uncensored' models, utilizing a 'Model Stock' merge method with a 32768 token context length. It features a clean tokenizer and embedding sizes, supporting a full 128K context size, and is optimized for stable performance in its intended use case.
Loading preview...
RicardoEstep/RPBizkit-v11-12B-Base Overview
RicardoEstep/RPBizkit-v11-12B-Base is a 12.2 billion parameter experimental model, developed by RicardoEstep, based on a "Mistral Nemo" architecture. It is constructed using a "Model Stock" merge method, combining ten different roleplay-focused models, with TheDrummer/Rocinante-X-12B-v1 serving as the base.
Key Capabilities & Features
- Roleplay Optimization: Specifically designed as a mix of "RP Uncensored" models, aiming for stable and effective performance in roleplay scenarios.
- Extended Context Window: Features a clean tokenizer and embedding sizes (131072), supporting an impressive 128K context size, patched from an initial 1M.
- Mistral Compatibility: Includes full Mistral chat template support.
- Merge Method: Utilizes the "Model Stock" merge method, as detailed in the provided configuration, to combine various specialized models.
Performance Insights
Rapid benchmarking using lm-evaluation-harness on an 8K context window with a Q8_0 quantized model yielded the following scores:
- Hellaswag: acc 56.25%, acc_norm 78.12%
- ifeval v4: inst_level_loose_acc 22.22%, inst_level_strict_acc 19.19%, prompt_level_loose_acc 15.62%, prompt_level_strict_acc 14.06%
Good for
- Developers and users seeking a specialized model for roleplay-centric applications.
- Experimentation with merged models and the "Model Stock" method.
- Use cases requiring a large context window (up to 128K tokens) for extended interactions.