teknium/llama-deus-7b-v3-lora-merged
teknium/llama-deus-7b-v3-lora-merged is a 7 billion parameter language model based on the Llama architecture, developed by teknium. This model is a LoRA-merged variant, meaning the LoRA weights are fully integrated, eliminating the need for separate merging. It was trained on an extensive and diverse dataset, including general, code, roleplay, and scientific instruction data, making it suitable for a wide range of conversational and instructional tasks.
Loading preview...
Overview
teknium/llama-deus-7b-v3-lora-merged is a 7 billion parameter language model built upon the Llama architecture. This model features a fully merged LoRA, which means users do not need to perform any additional merging steps to load and utilize it. It was developed by teknium and underwent 4 epochs of training over a single day, utilizing a rank 128 LoRA that targets attention heads, LM_Head, and MLP layers.
Key Capabilities
- Diverse Instruction Following: Trained on a broad spectrum of datasets including GPTeacher (General, Code, Roleplay), unreleased Roleplay V2, GPT4-LLM Uncensored, Unnatural Instructions, and WizardLM Uncensored.
- Scientific and Mathematical Reasoning: Enhanced with specialized datasets from CamelAI covering 20k Biology, 20k Physics, 20k Chemistry, and 50k Math GPT4 datasets.
- Code Generation: Includes CodeAlpaca in its training data, contributing to its ability in code-related tasks.
- Simplified Deployment: The LoRA is pre-merged, streamlining the model loading process.
Good for
- Applications requiring a versatile instruction-following model.
- Tasks involving roleplay and creative text generation.
- Educational or research purposes in biology, physics, chemistry, and mathematics.
- Code generation and understanding tasks.
- Developers seeking a Llama-based model with pre-integrated LoRA for ease of use.