mlabonne/Meta-Llama-3-12B-Instruct
mlabonne/Meta-Llama-3-12B-Instruct is a 15 billion parameter instruction-tuned language model, created by mlabonne through a merge of multiple NousResearch/Meta-Llama-3-8B-Instruct models. This model features an 8192-token context length and demonstrates improved performance over its base model, particularly in reasoning and general knowledge tasks. It is designed for conversational AI and instruction-following applications.
Loading preview...
Overview
mlabonne/Meta-Llama-3-12B-Instruct is a 15 billion parameter instruction-tuned language model. It was constructed by mlabonne using a "passthrough" merge method, combining five instances of the NousResearch/Meta-Llama-3-8B-Instruct model, each contributing different layer ranges. This merging strategy aims to enhance the model's overall capabilities and performance.
Key Capabilities
- Enhanced Instruction Following: The model is instruction-tuned, making it suitable for conversational agents and tasks requiring precise responses to prompts.
- Improved Reasoning: Evaluations show a notable improvement in reasoning benchmarks like AGIEval (41.7 vs 29.46) compared to the base Meta-Llama-3-12B model.
- General Knowledge: It exhibits stronger performance in general knowledge and truthfulness tasks, as indicated by scores on TruthfulQA and Bigbench.
- Context Length: Supports an 8192-token context window, allowing for processing longer inputs and maintaining conversational coherence over extended interactions.
Good for
- Developing chatbots and virtual assistants that require robust instruction adherence.
- Applications needing improved logical reasoning and factual accuracy.
- Tasks benefiting from a larger context window for more complex queries or multi-turn conversations.