Hugging Face Overview
Overview
Intellabs/sqft-phi-3-mini-4k-60-base is a 4 billion parameter language model developed by IntelLabs. It is a sparse version of the microsoft/Phi-3-mini-4k-instruct model, achieved by applying 60% sparsity using the Wanda pruning method. This model is part of the SQFT (Sparse Quantized Fine-Tuning) research initiative, focusing on low-cost model adaptation in low-precision sparse foundation models.
Key Characteristics
- Source Model: Based on Microsoft's Phi-3-mini-4k-instruct.
- Sparsity: Achieves 60% sparsity using the Wanda pruning technique.
- Parameter Count: 4 billion parameters.
- Context Length: Supports a context length of 4096 tokens.
- Research Focus: Developed under the SQFT framework, aimed at efficient deployment and adaptation of sparse models.
Use Cases
This model is particularly well-suited for scenarios requiring efficient, resource-aware deployment of language models. Its sparsity makes it a strong candidate for:
- Edge Devices: Deploying LLMs on hardware with limited computational resources.
- Research in Sparsity: Exploring the performance and efficiency trade-offs of highly sparse models.
- Cost-Effective Adaptation: Investigating low-cost methods for adapting foundation models.