Llama-3.1-Storm-8B: Enhanced 8B Generalist Model

Llama-3.1-Storm-8B is an 8 billion parameter model developed by Ashvini Kumar Jindal, Pawan Kumar Rajpoot, Ankur Parikh, and Akshita Sukhlecha. It is built on Meta AI's Llama-3.1-8B-Instruct and features a 32768 token context length. The model demonstrates significant performance improvements over its base model and Hermes-3-Llama-3.1-8B across a range of benchmarks.

Key Enhancements & Capabilities

This model's superior performance is attributed to a three-step process:

Self-Curation: Approximately 1 million high-quality examples were selected from 2.8 million open-source examples, focusing on educational value and difficulty, annotated using a Small Language Model (SLM).
Targeted Fine-tuning: Utilizes the Spectrum method, which accelerates training by selectively targeting 50% of layer modules based on their signal-to-noise ratio (SNR) and freezing the rest.
Model Merging: The fine-tuned model was merged with Llama-Spark using the SLERP method, blending characteristics from both parent models.

Performance Highlights

Llama-3.1-Storm-8B shows notable absolute gains over Meta-Llama-3.1-8B-Instruct:

Improved Instruction Following: IFEval Strict (+3.93%)
Enhanced Knowledge-Driven QA: GPQA (+7.21%), MMLU-Pro (+0.55%), AGIEval (+3.77%)
Better Reasoning: ARC-C (+3.92%), MuSR (+2.77%), BBH (+1.67%), AGIEval (+3.77%)
Superior Agentic Capabilities: BFCL Overall Acc (+7.92%), BFCL AST Summary (+12.32%)
Reduced Hallucinations: TruthfulQA (+9%)

Use Cases

This model is a powerful generalist, particularly useful for:

Applications requiring strong instruction following and reasoning.
Knowledge-driven question answering systems.
Function calling and agentic tasks, with impressive capabilities demonstrated on the BFCL benchmark.
Developers working with limited computational resources who need high performance from an 8B parameter model.