knifeayumu/Anubis-v1-Magnum-v4-SE-70B
knifeayumu/Anubis-v1-Magnum-v4-SE-70B is a 70 billion parameter language model created by knifeayumu through a SLERP merge of Doctor-Shotgun/L3.3-70B-Magnum-v4-SE and TheDrummer/Anubis-70B-v1. This model leverages the strengths of its constituent models to offer enhanced performance for general language tasks. With a 32768 token context length, it is suitable for applications requiring extensive contextual understanding.
Loading preview...
Model Overview
knifeayumu/Anubis-v1-Magnum-v4-SE-70B is a 70 billion parameter language model developed by knifeayumu. It was created using the SLERP merge method from MergeKit, combining two distinct pre-trained models to achieve a synergistic performance profile.
Merge Details
This model is a blend of:
- Doctor-Shotgun/L3.3-70B-Magnum-v4-SE
- TheDrummer/Anubis-70B-v1
The merge process utilized a specific configuration with varying t parameters across the models, indicating a nuanced weighting strategy to integrate their respective capabilities. The base model for the merge was TheDrummer/Anubis-70B-v1, and the process was conducted using bfloat16 precision.
Key Characteristics
- Architecture: A merged model, combining the strengths of two 70B parameter models.
- Parameter Count: 70 billion parameters, offering substantial capacity for complex language understanding and generation.
- Context Length: Supports a context window of 32768 tokens, enabling the processing of lengthy inputs and maintaining coherence over extended conversations or documents.
Potential Use Cases
Given its large parameter count and merged architecture, this model is well-suited for a variety of demanding NLP tasks, including advanced text generation, complex reasoning, and applications requiring deep contextual understanding over long sequences.