elastix-ai/HyperPrune-Llama-3.1-8B-2to4

TEXT GENERATIONPricing:Input $0.37 / Cached $0.074 / Output $0.38Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:8kTool Calling:SupportedPublished:Sep 4, 2026License:otherArchitecture:Transformer Featherless Exclusive Cold

HyperPrune-Llama-3.1-8B-2to4 is an 8 billion parameter language model developed by elastix-ai, derived from Meta's Llama-3.1-8B. This model has been pruned to 2:4 semi-structured sparsity using the HyperPrune method, achieving a 50% overall decoder sparsity. It is designed for efficient deployment in scenarios requiring reduced model size and faster inference while maintaining performance. This reproduction run focuses on evaluating the HyperPrune method within the BLADE sparsity comparison framework.

Loading preview...

HyperPrune-Llama-3.1-8B-2to4 Overview

This model is a 2:4 semi-structured sparse version of Meta's Llama-3.1-8B, developed by elastix-ai as part of the BLADE sparsity-method comparison. It utilizes the HyperPrune technique, resulting in approximately 50% overall decoder sparsity across all 32 decoder layers. The model is provided as plain sparse bf16/fp16 safetensors and is directly loadable with the transformers library.

Key Characteristics & Differences

  • Sparsity Method: Implements 2:4 semi-structured sparsity via HyperPrune, a method that learns sparsity using a shared and context-aware hypernetwork.
  • Calibration Corpus: Unlike the original HyperPrune paper, this reproduction uses DKYoon/SlimPajama-6B for calibration, aligning with the BLADE comparison protocol.
  • Full Decoder Pruning: All 32 decoder layers are pruned, achieving a true 50% sparsity, whereas the original HyperPrune recipe often leaves initial layers dense.
  • Hypernet Influence: The hypernetwork primarily determines the mask for the first 200 output rows of each projection, with remaining rows retaining the SparseGPT prior's mask.
  • Performance: Achieves a WikiText-2 word perplexity of 22.27 (BLADE's protocol) and a training wall-clock time of 24.2 minutes on an NVIDIA RTX PRO 6000 Blackwell GPU.

Use Cases

This model is suitable for applications where reduced memory footprint and faster inference are critical, without significant degradation in language understanding capabilities. It serves as a valuable benchmark for evaluating the effectiveness of semi-structured sparsity techniques on large language models.

Finetunes
1 models