plawanrath/gemma-2-9b-it-random-s70-pia
The plawanrath/gemma-2-9b-it-random-s70-pia model is a 9 billion parameter instruction-tuned Gemma-2 variant, created by Plawan Kumar Rath, that has undergone 70% random unstructured weight pruning. This model serves as a research artifact to study fairness degradation under weight pruning, specifically demonstrating bias amplification on the BBQ benchmark. It offers no storage or latency savings compared to the dense baseline model, as unstructured pruning does not reduce on-disk size or improve inference speed on typical hardware. Its primary use is for research into the effects of pruning on model bias, not for production deployment.
Loading preview...
Overview
This model, plawanrath/gemma-2-9b-it-random-s70-pia, is a 9 billion parameter instruction-tuned Gemma-2 variant developed by Plawan Kumar Rath. It is a research artifact specifically created to investigate the impact of weight pruning on model fairness. The model has undergone 70% random unstructured pruning, a process detailed in the paper "Weight Pruning Amplifies Bias: A Multi-Method Study of Compressed LLMs for Edge AI" (IEEE AIIoT 2026).
Key Characteristics
- Research Focus: Designed to study fairness degradation and bias amplification under weight pruning, particularly on the BBQ benchmark.
- Pruning Method: Utilizes uniform random unstructured pruning at a target sparsity of 70% across linear layers in transformer blocks.
- No Performance Benefits: Despite significant pruning, the model offers no storage savings (on-disk size is identical to the dense baseline) and no latency savings (inference speed is identical) on common hardware like Apple Silicon (MLX) or consumer GPUs, as unstructured sparsity is not exploited by dense GEMM kernels.
- Bias Amplification: The companion paper highlights that random pruning at this sparsity level induces measurable bias amplification, which may not be detectable by perplexity-based evaluations alone.
Good for
- Academic Research: Ideal for researchers studying the effects of model compression techniques, specifically weight pruning, on ethical AI concerns like bias and fairness.
- Understanding Pruning Limitations: Useful for demonstrating that not all pruning methods yield practical benefits in terms of model size or inference speed, especially unstructured pruning without specialized hardware or software support.
- Investigating Evaluation Metrics: Provides a case study for why perplexity alone can be an insufficient metric for evaluating pruned models, particularly concerning social biases.