anjohn0077/NEXS-qwen2.5-7b-multislerp
anjohn0077/NEXS-qwen2.5-7b-multislerp is a 7.6 billion parameter language model based on the Qwen2.5 architecture, specifically merged from Qwen/Qwen2.5-7B, Qwen/Qwen2.5-Coder-7B-Instruct, and Qwen/Qwen2.5-Math-7B. This model utilizes a multi-SLERP merge method to combine domain expertise in code generation and mathematical reasoning. It is designed to excel in tasks requiring both coding and advanced mathematical capabilities, offering a 32768 token context length.
Loading preview...
Overview
anjohn0077/NEXS-qwen2.5-7b-multislerp is a 7.6 billion parameter model derived from the Qwen2.5-7B base, specifically engineered for enhanced performance in code generation and mathematical reasoning. It is part of the NEXS multi-SLERP merge collection, leveraging the mergekit framework.
Key Capabilities
- Specialized Domain Expertise: This model integrates the strengths of Qwen/Qwen2.5-Coder-7B-Instruct and Qwen/Qwen2.5-Math-7B, making it proficient in tasks that require both coding and mathematical problem-solving.
- Multi-SLERP Merging: The model was created using a multi-SLERP (Spherical Linear Interpolation) method, which performs barycentric spherical interpolation on a hypersphere. This technique projects models into tangent space, interpolates, and projects back, effectively combining the 'task-vector space' deltas from the shared base model.
- Consistent Vocabulary: All merged variants share an identical vocabulary of 152064 tokens, eliminating the need for tokenizer reconciliation.
Good For
- Code Generation: Excels in generating and understanding code, benefiting from the Qwen2.5-Coder-7B-Instruct component.
- Mathematical Reasoning: Strong performance in mathematical tasks, inherited from the Qwen2.5-Math-7B component.
- Combined Tasks: Ideal for use cases that require a blend of coding and mathematical skills, such as scientific computing, algorithm development, or data analysis scripting.