LyraNovaHeart/Prismatic-12b-v0.1-Experimental-1115
LyraNovaHeart/Prismatic-12b-v0.1-Experimental-1115 is a 12 billion parameter language model, merged from Mistral-Nemo-Base-2407, MN-12b-Mag-Mell-R1, and Mistral-Nemo-Prism-12B-v5. This experimental model features a 32K context length and includes a fix for the ChatML format by incorporating an EOS token. It is designed for general language tasks, leveraging its merged architecture for potentially improved performance.
Loading preview...
Prismatic-12b-v0.1-Experimental-1115 Overview
LyraNovaHeart/Prismatic-12b-v0.1-Experimental-1115 is a 12 billion parameter experimental language model, developed by LyraNovaHeart. This model is a merge of several pre-trained language models, specifically mistralai/Mistral-Nemo-Base-2407, inflatebot/MN-12b-Mag-Mell-R1, and nbeerbower/Mistral-Nemo-Prism-12B-v5, utilizing the ties merge method via mergekit.
Key Features and Improvements
- ChatML Format Fix: A primary focus of this experimental release is to address issues with the ChatML format by correctly implementing an End-Of-Sequence (EOS) token, which was previously missing.
- Merged Architecture: The model benefits from a combination of different base models, aiming to leverage their respective strengths. The merge configuration weighted
inflatebot_MN-12B-Mag-Mell-R1at 0.3 andnbeerbower_Mistral-Nemo-Prism-12B-v5at 0.4, withmistralai_Mistral-Nemo-Base-2407serving as the base. - Context Length: It supports a substantial context length of 32,768 tokens, allowing for processing longer inputs and maintaining conversational coherence over extended interactions.
Intended Use Cases
This model is suitable for developers and researchers looking for an experimental 12B parameter model with an improved ChatML format. Its merged nature suggests potential for diverse general-purpose language generation and understanding tasks, particularly where the ChatML format is critical for interaction.