meta-llama/Llama-3.1-405B

Llama 3.1 405B: Multilingual Dialogue and Advanced Reasoning

The Meta Llama 3.1 405B is the largest model in the Llama 3.1 series, developed by Meta. This instruction-tuned, 405 billion parameter model is built on an optimized transformer architecture with Grouped-Query Attention (GQA) and boasts an extensive 128k token context length. It was trained on over 15 trillion tokens of publicly available online data, with a knowledge cutoff of December 2023.

Key Capabilities

• Multilingual Dialogue: Optimized for assistant-like chat in English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai, with support for additional languages through fine-tuning.
• High Performance: Outperforms many open-source and closed chat models on industry benchmarks, achieving 87.3% on MMLU and 96.8% on GSM-8K (CoT).
• Advanced Reasoning & Code: Demonstrates strong performance in reasoning tasks (e.g., 96.9% on ARC-C) and code generation (89.0% on HumanEval pass@1).
• Tool Use: Shows significant improvements in tool-use benchmarks like API-Bank (92.0%) and BFCL (88.5%).

Good For

• Commercial and Research Use: Intended for a wide range of applications, from assistant-like chat to natural language generation tasks.
• Multilingual Applications: Ideal for developing applications requiring robust performance across its 8 explicitly supported languages.
• System Integration: Designed to be deployed as part of larger AI systems, with Meta providing safeguards like Llama Guard 3, Prompt Guard, and Code Shield for responsible development.