alwoolley/Phi-4-mini-instruct
Phi-4-mini-instruct is a 3.8 billion parameter instruction-tuned decoder-only Transformer model developed by Microsoft, part of the Phi-4 family. It is built upon synthetic data and filtered public websites, focusing on high-quality, reasoning-dense data. With a 128K token context length, it excels in memory/compute-constrained environments and latency-bound scenarios, particularly for strong reasoning tasks like math and logic.
Loading preview...
Model Overview
Phi-4-mini-instruct is a 3.8 billion parameter instruction-tuned model from the Microsoft Phi-4 family, designed for efficiency and strong reasoning capabilities. It features a 128K token context length and a 200K vocabulary, supporting broad multilingual use. The model was trained on 5 trillion tokens, combining filtered public documents, high-quality educational data, code, and newly created synthetic "textbook-like" data focused on math, coding, common sense reasoning, and general knowledge. It underwent supervised fine-tuning and direct preference optimization for precise instruction adherence and robust safety.
Key Capabilities
- Efficient Reasoning: Optimized for strong reasoning, especially in math and logic, making it suitable for memory/compute-constrained and latency-bound environments.
- Multilingual Support: Features an expanded vocabulary and improved architecture for enhanced multilingual performance across 23 supported languages including Arabic, Chinese, English, French, German, Japanese, and Spanish.
- Instruction Following & Function Calling: Enhanced through advanced post-training techniques to improve instruction adherence and function calling capabilities.
- Robust Safety Measures: Incorporates safety post-training, evaluated through red teaming and adversarial simulations, showing resistance to jailbreak techniques across multiple languages.
Performance Highlights
Phi-4-mini-instruct demonstrates competitive performance against larger models in its class. For instance, it achieves 70.4 on BigBench Hard (0-shot, CoT), 67.3 on MMLU (5-shot), and 88.6 on GSM8K (8-shot, CoT). While strong in reasoning, its smaller size means it has limited factual knowledge, suggesting augmentation with search engines for RAG settings to mitigate factual incorrectness.
Intended Uses
This model is ideal for general-purpose AI systems and applications requiring strong reasoning in resource-limited settings. It serves as a valuable building block for generative AI features and accelerates research in language and multimodal models. Developers should consider its limitations, such as potential performance disparities across languages and factual inaccuracies, and implement appropriate safeguards for high-risk scenarios.