NemoMix-Unleashed-12B Overview

NemoMix-Unleashed-12B is a 12 billion parameter language model merge developed by MarinaraSpaghetti, utilizing the della_linear merge method with mistralaiMistral-Nemo-Base-2407 as its foundational model. This merge incorporates several Mistral-based models, including intervitens_mini-magnum-12b-v1.1, nbeerbower_mistral-nemo-bophades-12B, Sao10K_MN-12B-Lyra-v1, and nbeerbower_mistral-nemo-gutenberg-12B, alongside mistralaiMistral-Nemo-Instruct-2407.

Key Capabilities & Features

• Reduced Repetition: Engineered to exhibit less repetition, especially in longer contexts, addressing a common challenge in merged models.
• Enhanced Generative Quality: Aims to provide a more fluid and coherent generation experience compared to previous merge attempts.
• Optimized for Creative Tasks: Demonstrates strong performance in creative applications such as roleplay and storytelling.
• Mistral Instruct Format: Utilizes the Mistral Instruct chat format, with specific guidance provided for correct implementation.

Recommended Usage

• Temperature Settings: Recommended temperature range of 1.0-1.25, with 0.1 Top A or 0.01-0.1 Min P, and 0.8/1.75/2/0 DRY for optimal output.
• Context Length: Designed to perform well with its 32768 token context length, particularly benefiting from its repetition reduction at higher contexts.

This model is a result of extensive experimentation to refine merge techniques and improve generative consistency.