CY1232/Mistral-Nemo-12B-CY
CY1232/Mistral-Nemo-12B-CY is a 12.2 billion parameter language model created by CY1232, formed by merging Mistral-Nemo-Instruct-2407-abliterated and magnum-v4-12b using the SLERP method. This model leverages the strengths of its constituent models to provide a balanced performance across general language understanding and generation tasks. With a 32768 token context length, it is suitable for applications requiring extensive contextual awareness.
Loading preview...
Model Overview
CY1232/Mistral-Nemo-12B-CY is a 12.2 billion parameter language model developed by CY1232. It was created using the MergeKit tool, specifically employing the SLERP merge method to combine two distinct base models.
Merge Details
This model is a strategic merge of:
- natong19/Mistral-Nemo-Instruct-2407-abliterated
- anthracite-org/magnum-v4-12b
The merging process involved specific layer ranges (0 to 40 for both models) and parameter adjustments for self-attention and MLP layers, aiming to optimize the combined model's performance. The base model for the merge configuration was natong19/Mistral-Nemo-Instruct-2407-abliterated, and the tokenizer source was also derived from the base model.
Key Characteristics
- Parameter Count: 12.2 billion parameters
- Context Length: 32768 tokens
- Merge Method: Spherical Linear Interpolation (SLERP)
Use Cases
This model is designed for general-purpose language tasks, benefiting from the combined capabilities of its merged components. Its substantial context window makes it suitable for applications requiring processing and generating long-form content, complex reasoning, or detailed conversational interactions.