mergekit-community/MN-Nyx-Chthonia-12B
MN-Nyx-Chthonia-12B is a 12 billion parameter language model created by mergekit-community, merged using the Model Stock method with mistralai/Mistral-Nemo-Instruct-2407 as its base. This model integrates components from several specialized 12B models, including those focused on reasoning and psychology, to enhance its overall capabilities. It is designed for general-purpose language tasks, leveraging a diverse set of merged models for robust performance across various applications.
Loading preview...
Model Overview
MN-Nyx-Chthonia-12B is a 12 billion parameter language model developed by mergekit-community. It was created using the advanced Model Stock merge method, with mistralai/Mistral-Nemo-Instruct-2407 serving as the foundational base model. This merging approach combines the strengths of multiple pre-trained models to achieve a more versatile and capable LLM.
Key Merged Components
The model integrates several distinct 12B parameter models, each contributing unique characteristics. Notable components include:
DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESSmistralai/Mistral-Nemo-Base-2407redrix/sororicide-12B-Farer-Mell-Unslopmergekit-community/MN-Chthonia-12Byamatazen/EtherealAurora-12B-v2mergekit-community/MN-Anathema-12Bmergekit-community/MN-Ephemeros-12Bcombined with a LORA for reasoning and psychology (jtatman/mistral_nemo_12b_reasoning_psychology_lora)
Configuration and Tokenization
The merge utilized a specific YAML configuration, setting dtype to float32 and out_dtype to bfloat16. The tokenizer configuration sources tokens like <|im_start|>, <|im_end|>, [INST], and [/INST] from yamatazen/EtherealAurora-12B-v2 and mistralai/Mistral-Nemo-Instruct-2407 respectively, ensuring broad compatibility. The model is configured to use the chatml chat template.
Use Cases
Given its diverse merged components, MN-Nyx-Chthonia-12B is suitable for a wide range of general language generation and understanding tasks, potentially excelling in areas where the constituent models had strengths, such as reasoning and creative text generation.