Overview

The abacusai/Smaug-Qwen2-72B-Instruct is a 72.7 billion parameter instruction-tuned model, building upon the Qwen2-72B-Instruct architecture. It features a significant context length of 131,072 tokens, enabling it to process and understand extensive inputs.

Key Capabilities & Performance

This model distinguishes itself through enhanced performance in several critical areas compared to its base model:

• Reasoning: Achieves an overall score of 0.8241 on Big-Bench Hard (BBH), surpassing Qwen2-72B-Instruct's 0.8036, indicating stronger complex reasoning abilities.
• Code Generation: Demonstrates improved coding proficiency with a Pass@1 score of 0.3357 on LiveCodeBench, compared to 0.3139 for the base model.
• General Instruction Following: Scores 48.0 on Arena-Hard, outperforming Qwen2-72B-Instruct's 43.5, suggesting better overall instruction adherence and helpfulness in competitive scenarios.

Use Cases

Given its strengths, Smaug-Qwen2-72B-Instruct is particularly well-suited for:

• Applications requiring advanced logical reasoning and problem-solving.
• Code generation and understanding tasks where accuracy is paramount.
• Complex conversational agents or assistants that need to handle intricate instructions and maintain context over long interactions.