jeiku/Nitrals_Monster_7B
jeiku/Nitrals_Monster_7B is a 7 billion parameter language model created by jeiku through a SLERP merge of several pre-trained models, including Test157t/Heracleana-Maid-7b and cognitivecomputations/samantha-1.1-westlake-7b. This model leverages a 4096-token context length and is designed to combine the strengths of its constituent models. It is suitable for general language generation tasks, reflecting the diverse capabilities of its merged components.
Loading preview...
Overview
jeiku/Nitrals_Monster_7B is a 7 billion parameter language model developed by jeiku. It was created using the SLERP merge method via mergekit, combining the capabilities of multiple base models. This approach aims to synthesize the strengths of its constituent models into a single, more versatile model.
Merge Details
The model integrates components from:
Test157t/Heracleana-Maid-7bTest157t/Heracleana-Maid-7bcombined withjeiku/Futadom_Mistralcognitivecomputations/samantha-1.1-westlake-7bcombined withjeiku/Humiliation_Mistral
Configuration
The merge process utilized a specific YAML configuration, applying varying t parameters across self_attn and mlp layers, with bfloat16 dtype. This detailed configuration suggests an effort to fine-tune the contribution of each merged model's layers.
Potential Use Cases
Given its merged nature, Nitrals_Monster_7B is likely suitable for a range of general-purpose language generation and understanding tasks, benefiting from the diverse training data and architectures of its base models. Its 4096-token context length supports processing moderately long inputs.