Hugging Face Overview
Overview
The abacusai/Smaug-Qwen2-72B-Instruct is a 72.7 billion parameter instruction-tuned model, building upon the Qwen2-72B-Instruct architecture. It features a significant context length of 131,072 tokens, enabling it to process and understand extensive inputs.
Key Capabilities & Performance
This model distinguishes itself through enhanced performance in several critical areas compared to its base model:
- Reasoning: Achieves an overall score of 0.8241 on Big-Bench Hard (BBH), surpassing Qwen2-72B-Instruct's 0.8036, indicating stronger complex reasoning abilities.
- Code Generation: Demonstrates improved coding proficiency with a Pass@1 score of 0.3357 on LiveCodeBench, compared to 0.3139 for the base model.
- General Instruction Following: Scores 48.0 on Arena-Hard, outperforming Qwen2-72B-Instruct's 43.5, suggesting better overall instruction adherence and helpfulness in competitive scenarios.
Use Cases
Given its strengths, Smaug-Qwen2-72B-Instruct is particularly well-suited for:
- Applications requiring advanced logical reasoning and problem-solving.
- Code generation and understanding tasks where accuracy is paramount.
- Complex conversational agents or assistants that need to handle intricate instructions and maintain context over long interactions.