The jan-ai/Solar-10.7B-SLERP is a 10.7 billion parameter language model created by Jan using the SLERP merge method. It combines the strengths of upstage/SOLAR-10.7B-Instruct-v1.0 and janhq/Pandora-v1-10.7B, leveraging their respective capabilities. This model is designed for general language tasks, offering a balanced performance profile derived from its merged base models. It operates with a context length of 4096 tokens, making it suitable for various conversational and text generation applications.
Loading preview...
Model Overview
The jan-ai/Solar-10.7B-SLERP is a 10.7 billion parameter language model developed by Jan. This model is a product of a strategic merge using the SLERP (Spherical Linear Interpolation) method, combining two high-performing models from the OpenLLM Leaderboard as of December 14th:
- upstage/SOLAR-10.7B-Instruct-v1.0
- janhq/Pandora-v1-10.7B
The base model for this merge is upstage/SOLAR-10.7B-Instruct-v1.0. The SLERP method was applied across all 48 layers of both source models, with specific parameter weighting for self-attention and MLP layers to optimize performance.
Key Characteristics
- Merged Architecture: Leverages the strengths of two distinct 10.7B models to create a more robust and versatile language model.
- SLERP Method: Utilizes a sophisticated merging technique to blend model weights effectively, aiming for improved overall performance.
- Prompt Format: Employs the ChatML prompt template, facilitating structured conversational interactions.
Intended Use Cases
This model is suitable for a variety of applications, particularly those benefiting from a merged model's balanced capabilities. It can be run locally using the Jan application, which offers an open-source, offline, and OpenAI-compatible environment. Developers can utilize the provided GGUF version for local deployment and experimentation.