Model Overview

MaziyarPanahi/calme-2.2-llama3-70b is a 70 billion parameter instruction-tuned language model, developed by MaziyarPanahi. It is a fine-tuned (DPO) version of the meta-llama/Meta-Llama-3-70B-Instruct base model, previously known as MaziyarPanahi/Llama-3-70B-Instruct-DPO-v0.2.

Key Capabilities & Performance

This model exhibits strong performance across a range of benchmarks, as evaluated on the Open LLM Leaderboard. Its key strengths include:

• Reasoning: Achieves 72.53 on AI2 Reasoning Challenge (25-Shot) and 80.41 on MMLU (5-Shot).
• Common Sense: Scores 86.22 on HellaSwag (10-Shot) and 82.79 on Winogrande (5-shot).
• Mathematical Reasoning: Demonstrates capability with 88.25 on GSM8k (5-shot).
• Overall Performance: Boasts an average score of 78.96 across various metrics, indicating robust general-purpose instruction following.

Usage and Prompt Format

The model utilizes the ChatML prompt template, making it compatible with standard conversational AI frameworks. Quantized GGUF versions are also available for efficient deployment.

Good for

• General instruction-following tasks.
• Conversational AI and chatbot applications.
• Reasoning and problem-solving scenarios.
• Applications requiring robust language generation based on the Llama 3 architecture.