FelixChao/Severus-7B
FelixChao/Severus-7B is a 7 billion parameter language model created by FelixChao, formed by merging samir-fama/FernandoGPT-v1 and FelixChao/NinjaDolphin-7B. This model leverages a unique 'passthrough' merge method across specific layer ranges of its constituent models. It is designed for general text generation tasks, offering a 4096 token context length.
Loading preview...
Severus-7B: A Merged Language Model
Severus-7B is a 7 billion parameter language model developed by FelixChao, distinguished by its unique construction as a merge of two existing models:
- samir-fama/FernandoGPT-v1
- FelixChao/NinjaDolphin-7B
Key Characteristics
This model utilizes a specific 'passthrough' merge method, combining layers from its base models in a structured manner:
- Layers 0-24 are sourced from
samir-fama/FernandoGPT-v1. - Layers 24-32 are sourced from
FelixChao/NinjaDolphin-7B.
This configuration aims to integrate the strengths of both foundational models. The model operates with a float16 data type for efficiency.
Usage and Application
Severus-7B is suitable for various text generation tasks, leveraging its 7 billion parameters and 4096 token context length. Developers can easily integrate it using the Hugging Face transformers library for tasks such as:
- Answering questions
- Generating creative text
- Engaging in conversational AI
The provided usage example demonstrates how to load the model and tokenizer, apply a chat template, and generate text with specified parameters like max_new_tokens, temperature, top_k, and top_p.