RicardoEstep/RPBizkit-v6-12B

RicardoEstep/RPBizkit-v6-12B Overview

RPBizkit-v6-12B is a 12.2 billion parameter experimental merge model developed by RicardoEstep. It was created using a multi-stage merging process involving Karcher-Mean and DARE TIES techniques, along with a custom Python script, to combine several "RP Uncensored" models. The primary goal of this version is to address the "Beige Effect" observed in its predecessor, which resulted in uncreative and uninitiative outputs, by re-integrating the original models that formed previous merges.

Key Capabilities & Merging Process

The model's construction is divided into six parts, each contributing to its overall characteristics:

• Intelligence Core: Merged using Karcher-Mean from models like DreadPoor/Krix-12B-Model_Stock and romaingrx/red-teamer-mistral-nemo.
• Narrative Core: Merged using DARE TIES from models such as ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2 and allura-org/MN-12b-RP-Ink, focusing on narrative generation.
• Dark Style Core: Also merged with DARE TIES, incorporating models like DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESS and ChaoticNeutrals/Nera_Noctis-12B to influence stylistic output.
• Mixing the Cores: A subsequent DARE TIES merge combines the Intelligence, Narrative, and Dark Style cores.
• Light Re-Abiliteration: Applies a LoRA (nbeerbower/Mistral-Nemo-12B-abliterated-LORA) with hybrid scaling to the merged cores for stability.
• Light Re-UnSlop: A final DARE TIES merge with TheDrummer/UnslopNemo-12B-v4.1 to further enhance stability.

Usage Recommendations

This model features a clean tokenizer and embedding sizes (131072) based on Mistral. However, it is explicitly noted that the model may drift if standard ChatML or Mistral-based chat templates are used. The recommended chat template is Alpaca with "RAW" inputs. While the model is theoretically capable of supporting a 128K context size, the recommended maximum context size for stable performance is 8K (8192) tokens.