Masterjp123/NeuralMaid-7b
Masterjp123/NeuralMaid-7b is a 7 billion parameter language model created by Masterjp123 through a merge of NurtureAI/neural-chat-7b-v3-16k and NeverSleep/Noromaid-7b-v0.2 using the SLERP method. This model combines characteristics from its base models, offering a 4096 token context length. It is designed as a general-purpose merged model, leveraging the strengths of its components.
Loading preview...
Masterjp123/NeuralMaid-7b: A Merged Language Model
Masterjp123/NeuralMaid-7b is a 7 billion parameter language model resulting from a merge of two pre-trained models: NurtureAI/neural-chat-7b-v3-16k and NeverSleep/Noromaid-7b-v0.2. This model was created using the SLERP merge method via the MergeKit tool.
Key Characteristics
- Architecture: A 7B parameter model, inheriting its base architecture from the merged components.
- Context Length: Supports a context window of 4096 tokens.
- Merge Method: Utilizes the Spherical Linear Interpolation (SLERP) method for combining model weights, specifically applying different interpolation ratios to self-attention and MLP layers.
- Component Models: Integrates features from both
neural-chat-7b-v3-16k(known for its instruction-following capabilities and larger context) andNoromaid-7b-v0.2.
Intended Use
This model is suitable for general language generation tasks, leveraging the combined strengths of its constituent models. Developers can explore its performance across various NLP applications where a 7B parameter model with a 4k context window is appropriate.