RicardoEstep/RPBizkit-v8-12B
RicardoEstep/RPBizkit-v8-12B is a 12.2 billion parameter experimental Mistral Nemo mix, created by RicardoEstep, utilizing Model Stock and SLERP merge methods with Mergekit. This model is specifically designed as an "RP Uncensored" mix, focusing on roleplay capabilities. It supports a 32768-token context length and is optimized for stable and consistent roleplay generation.
Loading preview...
RicardoEstep/RPBizkit-v8-12B: A Stable Roleplay-Optimized Merge
RicardoEstep/RPBizkit-v8-12B is a 12.2 billion parameter experimental language model, representing the creator's most stable iteration of a "Mistral Nemo" mix. This model is constructed using advanced merging techniques, specifically Model Stock and SLERP (Spherical Linear Interpolation) with Mergekit, to combine various fine-tuned models.
Key Capabilities & Merging Strategy
The model's architecture is a three-part merging process, designed to enhance roleplay (RP) capabilities:
- Part One: "RP Core": Merges several RP-focused Mistral-Nemo models, including
natong19/Mistral-Nemo-Instruct-2407-abliteratedas the base,TheDrummer/UnslopNemo-12B-v4.1,ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2,allura-org/MN-12b-RP-Ink, andnbeerbower/mistral-nemo-gutenberg-12B-v4. This stage prioritizes RP logic. - Part Two: "Substance Core": Combines additional models like
SicariusSicariiStuff/Impish_Bloodmoon_12B,ChaoticNeutrals/Nera_Noctis-12B,allura-org/Bigger-Body-12b, andReadyArt/Forgotten-Safeword-12B-v4.0with the same base, focusing on vocabulary and content generation. - Part Three: "The Final Mix": Utilizes SLERP to blend the "RP Core" and "Substance Core" outputs. This final merge applies specific weighting to self-attention layers (prioritizing RP Core logic) and MLP layers (prioritizing Substance Core vocabulary), resulting in a balanced model.
Technical Specifications & Usage
- Context Length: The model is designed to support a full 128K context size, with clean tokenizer and embedding sizes (131072).
- Recommended Chat Template: The Alpaca chat template with "RAW" inputs is recommended for optimal performance, though it also supports an optional Mistral V3 chat template.
Good for
- Roleplay (RP) Applications: Specifically designed and optimized for generating "RP Uncensored" content.
- Experimental Merging: Demonstrates advanced model merging techniques for combining specialized fine-tunes.