Overview
Undi95/Lumimaid-Magnum-v4-12B is a 12 billion parameter language model created by Undi95, built upon a merge of the Lumimaid and Magnum v4 models. It incorporates a finetune specifically on Claude input, trained with a 16k context window, which is integrated using the DELLA merge method in Mergekit. The model supports a substantial context length of 32,768 tokens.
Key Capabilities
- Merged Architecture: Combines the strengths of Lumimaid and Magnum v4, potentially offering a broader range of capabilities than its base models individually.
- Claude Input Finetuning: Enhanced with a finetune on Claude-style inputs, which may improve its ability to follow complex instructions or generate responses in a conversational style.
- Extended Context Window: Features a 32,768 token context length, allowing for processing and generating longer texts while maintaining coherence.
- Mistral Prompt Template: Utilizes the Mistral prompt template (
<s>[INST] {input} [/INST] {output}</s>), making it compatible with common instruction-following formats.
Good For
- General Text Generation: Suitable for a wide array of text generation tasks due to its merged base models and instruction-following capabilities.
- Instruction Following: The Claude input finetuning suggests improved performance in responding to prompts and instructions effectively.
- Long-form Content: Its 32,768 token context length makes it well-suited for applications requiring understanding or generation of extensive documents or conversations.