Overview

Intellabs/sqft-phi-3-mini-4k-60-base is a 4 billion parameter language model developed by IntelLabs. It is a sparse version of the microsoft/Phi-3-mini-4k-instruct model, achieved by applying 60% sparsity using the Wanda pruning method. This model is part of the SQFT (Sparse Quantized Fine-Tuning) research initiative, focusing on low-cost model adaptation in low-precision sparse foundation models.

Key Characteristics

• Source Model: Based on Microsoft's Phi-3-mini-4k-instruct.
• Sparsity: Achieves 60% sparsity using the Wanda pruning technique.
• Parameter Count: 4 billion parameters.
• Context Length: Supports a context length of 4096 tokens.
• Research Focus: Developed under the SQFT framework, aimed at efficient deployment and adaptation of sparse models.

Use Cases

This model is particularly well-suited for scenarios requiring efficient, resource-aware deployment of language models. Its sparsity makes it a strong candidate for:

• Edge Devices: Deploying LLMs on hardware with limited computational resources.
• Research in Sparsity: Exploring the performance and efficiency trade-offs of highly sparse models.
• Cost-Effective Adaptation: Investigating low-cost methods for adapting foundation models.