Overview
teknium/llama-deus-7b-v3-lora-merged is a 7 billion parameter language model built upon the Llama architecture. This model features a fully merged LoRA, which means users do not need to perform any additional merging steps to load and utilize it. It was developed by teknium and underwent 4 epochs of training over a single day, utilizing a rank 128 LoRA that targets attention heads, LM_Head, and MLP layers.
Key Capabilities
- Diverse Instruction Following: Trained on a broad spectrum of datasets including GPTeacher (General, Code, Roleplay), unreleased Roleplay V2, GPT4-LLM Uncensored, Unnatural Instructions, and WizardLM Uncensored.
- Scientific and Mathematical Reasoning: Enhanced with specialized datasets from CamelAI covering 20k Biology, 20k Physics, 20k Chemistry, and 50k Math GPT4 datasets.
- Code Generation: Includes CodeAlpaca in its training data, contributing to its ability in code-related tasks.
- Simplified Deployment: The LoRA is pre-merged, streamlining the model loading process.
Good for
- Applications requiring a versatile instruction-following model.
- Tasks involving roleplay and creative text generation.
- Educational or research purposes in biology, physics, chemistry, and mathematics.
- Code generation and understanding tasks.
- Developers seeking a Llama-based model with pre-integrated LoRA for ease of use.