elastix-ai/HyperPrune-Llama-2-7B-2to4

HyperPrune-Llama-2-7B-2to4 Overview

This model is a Llama-2-7B variant from Meta, specifically pruned to achieve 2:4 semi-structured sparsity using the HyperPrune method. Developed by elastix-ai, this checkpoint is a reproduction run for the BLADE sparsity-method comparison, focusing on evaluating the HyperPrune technique under specific conditions.

Key Characteristics & Differences

• Sparsity: Achieves 50.05% overall decoder sparsity, with all 32 decoder layers pruned to the 2:4 ratio. This differs from the original HyperPrune paper's recipe, which kept the first two layers dense.
• Calibration Corpus: Uses DKYoon/SlimPajama-6B (validation split) as its calibration corpus, aligning with BLADE's protocol, instead of allenai/c4 used in the original paper.
• Hypernet Influence: Approximately 3.5% of the mask was determined by the hypernet, with the remaining mask derived from the SparseGPT prior, reflecting HyperPrune's default fixed_rows_count setting.
• Performance: Achieves a WikiText-2 word perplexity of 21.65 (compared to a dense reference of 9.19 under the same protocol). Training completed in 22.1 minutes with a peak GPU usage of 8.82 GB.

Use Cases

This model is primarily valuable for researchers and developers interested in:

• Evaluating semi-structured sparsity: Specifically, the HyperPrune method's effectiveness on Llama-2-7B.
• Benchmarking: As part of the BLADE comparison, it provides a standardized sparse model for performance and efficiency analysis.
• Exploring model compression: Understanding the trade-offs between sparsity levels and model performance in a practical setting.