Xiaojian9992024/Phi-4-mini-UNOFFICAL
Xiaojian9992024/Phi-4-mini-UNOFFICAL is a 14.7 billion parameter language model created by Xiaojian9992024, derived from a partial merge of Microsoft's Phi-4 model. This model utilizes a passthrough merge method, specifically incorporating only the first 8 layers of the original Phi-4 architecture, resulting in a smaller, more efficient variant. It is designed for applications requiring a compact yet capable model, offering a balance between performance and resource consumption with a context length of 32768 tokens.
Loading preview...
Overview
This model, Xiaojian9992024/Phi-4-mini-UNOFFICAL, is a 14.7 billion parameter language model created by Xiaojian9992024. It is a specialized variant derived from Microsoft's Phi-4 model, specifically engineered using a 'passthrough' merge method. This approach selectively incorporates only the initial 8 layers of the original Phi-4 architecture, resulting in a significantly more compact model while retaining core capabilities.
Key Characteristics
- Derived from Phi-4: Built upon the foundational architecture of Microsoft's Phi-4 model.
- Reduced Footprint: Achieves a smaller parameter count (14.7B) by utilizing only the first 8 layers of the base model, making it more efficient for deployment in resource-constrained environments.
- Passthrough Merge: Employs a direct layer selection method rather than complex blending, ensuring architectural integrity from the source model.
- Context Length: Supports a substantial context window of 32768 tokens, suitable for processing longer inputs and generating coherent, extended outputs.
Good for
- Applications requiring a capable language model with a reduced memory footprint compared to full-sized models.
- Scenarios where the core functionalities of Phi-4 are needed but with improved inference speed or lower computational cost.
- Experiments and development focusing on the performance of truncated large language models.