MaximalSlerp: A Merged 13B Language Model
MaximalSlerp is a 13 billion parameter language model developed by Brouz. It is the result of a Gradient Slerp merge operation, combining two distinct base models: Gryphe/MythoLogic-L2-13b and The-Face-Of-Goonery/Huginn-13b-v1.2. This merging process was executed using the Mergekit framework, specifically its YAML branch.
Key Characteristics
- Architecture: A blend of two 13B models, MythoLogic-L2-13b and Huginn-13b-v1.2, through a Gradient Slerp merge.
- Parameter Count: 13 billion parameters.
- Context Length: Supports a context window of 4096 tokens.
- Merging Method: Utilizes the Mergekit tool for its creation, indicating a focus on combining and enhancing existing model strengths.
Potential Use Cases
- General Text Generation: Suitable for a wide array of creative and factual text generation tasks, benefiting from the combined knowledge of its base models.
- Exploration of Merged Models: Ideal for developers and researchers interested in the performance characteristics and emergent capabilities of models created via advanced merging techniques like Gradient Slerp.
- Applications requiring a 13B model: Can be deployed in scenarios where a medium-sized, capable language model is required, potentially offering a unique performance profile compared to its constituents.