RicardoEstep/RPBizkitRemiX-v1-12B
RicardoEstep/RPBizkitRemiX-v1-12B is a 12 billion parameter experimental language model created by RicardoEstep, built by merging several of RicardoEstep's previous models using the Model Stock merge method. This model features a clean tokenizer and embedding sizes (131072) based on Mistral, and is designed for stable performance with a recommended Alpaca chat template. It offers a 32768 token context length, though an 8K context is recommended due to LLaMA 3 limitations, making it suitable for general text generation tasks.
Loading preview...
Model Overview
RicardoEstep/RPBizkitRemiX-v1-12B is an experimental 12 billion parameter language model developed by RicardoEstep. It was created by merging multiple prior models from RicardoEstep's collection, specifically using the "Model Stock" merge method and Mergekit, with RicardoEstep/RPBizkit-v6-12B serving as the base model. The merge process aimed to give equal importance to the data from all included models.
Key Characteristics
- Architecture: A unique merge of several 12B models, including AngelSlayerKrix-12B, RPBizkit-v2-12B, RPBizkit-v4-12B, RPBizkit-v4-12B_Lorablated, RPBizkit-v5-12B-Lorablated, and RPBizkit-v6-12B.
- Tokenizer: Features a "Clean Tokenizer & Embedding Sizes (131072)" based on Mistral, contributing to its stability.
- Context Length: While designed to support a 128K (131072) context size, the recommended maximum context for stable performance is 8K (8192) due to underlying LLaMA 3 limitations.
- Chat Template: The model is configured to not use any default chat template, with the "Alpaca (with RAW inputs)" template being the recommended approach to avoid drifting.
Use Cases
This model is suitable for general text generation and conversational AI where a stable, merged model with a clean tokenizer is beneficial. Its experimental nature suggests it's ideal for users looking to explore the capabilities of custom-merged models and those who prefer specific chat template control.