Overview

automerger/Inex12Yamshadow-7B is a 7 billion parameter language model developed by Maxime Labonne. It is an automated merge of two distinct models: MSL7/INEX12-7b and automerger/YamShadow-7B.

Key Capabilities

• Automated Merge Architecture: Utilizes a slerp (spherical linear interpolation) merge method to combine the weights of its base models.
• Parameter Specific Merging: The merge configuration applies varying t values for self_attn and mlp layers, suggesting a fine-tuned approach to integrate the strengths of both source models.
• General Language Generation: Suitable for a wide range of text generation tasks, leveraging its 7 billion parameters and 4096-token context window.

Good For

• Experimentation with Merged Models: Developers interested in exploring the performance characteristics of models created via automated merging techniques.
• General Purpose Text Generation: Can be used for various applications requiring text completion, question answering, or content creation based on its foundational models.
• Understanding Merge Strategies: Provides a practical example of a slerp merge with specific layer-wise parameter adjustments.