RicardoEstep/RPBizkit-v6-12B
RicardoEstep/RPBizkit-v6-12B is a 12.2 billion parameter experimental merge model created by RicardoEstep, utilizing Karcher-Mean and DARE TIES merging techniques with Mergekit and custom Python scripting. This model is a mix of several "RP Uncensored" models, specifically designed to overcome the "Beige Effect" of prior versions by integrating distinct "Intelligence Core," "Narrative Core," and "Dark Style Core" components. It features a clean tokenizer and embedding sizes (131072) based on Mistral, optimized for roleplay and creative text generation, with a recommended maximum context size of 8K tokens.
Loading preview...
RicardoEstep/RPBizkit-v6-12B Overview
RicardoEstep/RPBizkit-v6-12B is a 12.2 billion parameter experimental merge model developed by RicardoEstep. It was created using a combination of Karcher-Mean and DARE TIES merging techniques, facilitated by Mergekit and a custom Python script. This version specifically addresses the "Beige Effect" observed in previous iterations, which resulted in uncreative and uninitiative outputs.
Key Capabilities & Design
- Multi-Core Architecture: The model is built from six distinct parts, including an "Intelligence Core" (merged with Karcher-Mean), a "Narrative Core" (merged with DARE TIES), and a "Dark Style Core" (merged with DARE TIES), each contributing to different aspects of its generation capabilities.
- Hybrid LoRA Integration: It incorporates a "Light Re-Abiliteration" step using a custom Python script to apply a LoRA (nbeerbower/Mistral-Nemo-12B-abliterated-LORA) with hybrid scaling for attention and MLP layers, followed by a "Light Re-UnSlop" step using DARE TIES for stability.
- Tokenizer & Context: Features a clean tokenizer and embedding sizes (131072) based on Mistral. While designed to support a 128K context, the recommended maximum context size for stable performance is 8K (8192) tokens.
- Chat Template: It is configured to avoid ChatML or Mistral chat templates, with the recommended template being "Alpaca (with "RAW" inputs)" to prevent model drift.
Good for
- Roleplay and Creative Text Generation: The model is a mix of "RP Uncensored" models, making it suitable for generating diverse and engaging roleplay scenarios and creative narratives.
- Experimental Merging Techniques: Developers interested in advanced model merging strategies (Karcher-Mean, DARE TIES) and custom LoRA application will find its architecture insightful.
- Overcoming Generative Monotony: Designed to combat the "Beige Effect," it aims to produce more creative and initiative responses compared to prior versions.