nota-ai/st-llama-1-5.5b-ppl
The nota-ai/st-llama-1-5.5b-ppl model, developed by Nota AI, is a 5.5 billion parameter depth-pruned version of the LLaMA-1-7B model. This model is optimized for efficient text generation through a one-shot pruning method based on perplexity (PPL) and light LoRA-based retraining. It is designed for research and non-commercial projects requiring a more compact yet capable language model derived from the LLaMA architecture.
Loading preview...
Overview
This model, nota-ai/st-llama-1-5.5b-ppl, is a 5.5 billion parameter variant of the LLaMA-1-7B model, developed by Nota AI. It is part of the "Shortened LLaMA" series, which focuses on creating more efficient large language models through a novel depth-pruning technique. The model achieves a 20% reduction in parameters from its LLaMA-1-7B base by identifying and pruning unimportant Transformer blocks, followed by light LoRA-based retraining.
Key Capabilities & Features
- Depth Pruning: Utilizes a one-shot pruning method based on perplexity (PPL) to reduce model size while aiming to retain performance.
- Efficiency: Designed for more efficient text generation compared to its larger base model.
- Research Focus: Intended primarily for research and non-commercial applications.
- Derived from LLaMA: Benefits from the foundational architecture of the original LLaMA models.
Use Cases
- Efficient LLM Deployment: Suitable for scenarios where computational resources or inference speed are critical.
- Research & Experimentation: Ideal for researchers exploring model compression techniques and their impact on performance.
- Non-commercial Projects: Can be integrated into non-commercial applications requiring a compact LLaMA-based model.
For more technical details, refer to the associated paper: Shortened LLaMA: A Simple Depth Pruning for Large Language Models.