grimjim/Llama-Nephilim-Metamorphosis-v2-8B
The grimjim/Llama-Nephilim-Metamorphosis-v2-8B is an 8-billion parameter language model created by grimjim through a SLERP merge of Meta Llama 3.1-8B-Instruct and grimjim/llama-3-Nephilim-v3-8B. This model is designed for coherent direct chat and text generation, retaining the long context capabilities of Llama 3.1. It is optimized for general conversational AI and text generation tasks, with tested context handling up to 16K tokens.
Loading preview...
Overview
Llama-Nephilim-Metamorphosis-v2-8B is an 8-billion parameter language model developed by grimjim, created by merging a Llama 3.1 Instruct model with a Llama 3 fine-tune (grimjim/llama-3-Nephilim-v3-8B). This merge was performed using the SLERP method, specifically designed to produce a coherent model for direct chat and text generation while preserving the long context capabilities inherent in Llama 3.1.
Key Capabilities
- Coherent Text Generation: Optimized for producing natural and consistent responses in chat and general text generation scenarios.
- Long Context Handling: Retains the ability to process and understand long contexts, with testing confirming performance up to 16K tokens.
- Llama 3.1 Foundation: Benefits from the robust architecture and safety features of the Llama 3.1 Instruct base model.
- Gradient Merge Strategy: Utilizes a gradient merge with specific layer retention (embed_tokens and lm_head from 3.1) to enhance context handling, particularly beyond 8K tokens.
Good for
- General Chatbots: Ideal for conversational AI applications requiring coherent and context-aware interactions.
- Creative Writing & Content Generation: Suitable for generating various forms of text, from stories to articles.
- Applications Requiring Extended Context: Useful in scenarios where maintaining understanding over long passages of text is crucial, such as summarization or detailed Q&A.