Model Overview

NousResearch/Meta-Llama-3-70B is a 70 billion parameter instruction-tuned large language model developed by Meta. It utilizes an optimized transformer architecture and features an 8k token context length. This model is specifically optimized for dialogue use cases, demonstrating strong performance across various industry benchmarks compared to other open-source chat models.

Key Capabilities

• Enhanced Performance: Significantly outperforms its predecessor, Llama 2 70B, across a range of benchmarks including MMLU (82.0 vs 52.9), GPQA (39.5 vs 21.0), HumanEval (81.7 vs 25.6), GSM-8K (93.0 vs 57.5), and MATH (50.4 vs 11.6).
• Optimized for Dialogue: Instruction-tuned using supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety, making it ideal for assistant-like chat.
• Robust Training: Pretrained on over 15 trillion tokens of publicly available online data, with the 70B model's data cutoff in December 2023. Fine-tuning included over 10 million human-annotated examples.
• Reduced Refusals: Features improved fine-tuning to significantly reduce false refusals to benign prompts, enhancing user experience and helpfulness.

Intended Use Cases

• Commercial and Research Applications: Suitable for a wide array of commercial and research endeavors requiring advanced natural language generation.
• Assistant-like Chat: Optimized for conversational AI and dialogue systems.
• Code Generation: Demonstrates strong performance in coding tasks, as indicated by its HumanEval score.
• General Reasoning and Knowledge: Excels in general knowledge, common sense reasoning, and reading comprehension tasks.