MSL7/Liph.42-slerp is a 3 billion parameter language model created by MSL7, built using a series of slerp merges. This model integrates components from various 'liminerity' and 'rhysjones' models, including liminerity/merge5 and liminerity/Phigments12, to combine their strengths. It is designed for general text generation tasks, leveraging its merged architecture to potentially offer a balanced performance profile within its 2048-token context window.
Loading preview...
Model Overview
MSL7/Liph.42-slerp is a 3 billion parameter language model developed by MSL7. It is constructed through a complex series of spherical linear interpolation (slerp) merges, utilizing the mergekit tool. The model integrates layers from several base models, primarily from the liminerity and rhysjones collections, such as liminerity/merge5, liminerity/Phigments12, rhysjones/phi-2-orange-v2, and others.
Merge Configuration
The model's architecture is a result of multiple slerp merge operations, where different layers (specifically self_attn and mlp filters) from various source models are combined with specific t values. This fine-grained merging approach aims to blend the characteristics of its constituent models. The base models involved in the final merge include liminerity/merge5 and liminerity/Phigments12.
Potential Use Cases
Given its merged nature and 3 billion parameters, Liph.42-slerp is suitable for a range of general-purpose natural language processing tasks. Its 2048-token context window supports moderate-length text generation and understanding. Developers might find it useful for applications requiring a compact yet capable model, potentially excelling in areas where its merged components individually showed strength.