ausascsdas/phi-4
phi-4 is a 14.7 billion parameter dense decoder-only Transformer model developed by Microsoft Research. It is trained on 9.8 trillion tokens, including synthetic datasets, filtered public domain websites, and academic books, with a 16K token context length. The model is optimized for high-quality data and advanced reasoning, making it suitable for memory/compute constrained environments and latency-bound scenarios requiring strong logical capabilities.
Loading preview...
Model Overview
Microsoft Research's phi-4 is a 14.7 billion parameter dense decoder-only Transformer model, trained on 9.8 trillion tokens with a 16K token context length. It was developed using a blend of synthetic datasets, filtered public domain websites, and acquired academic books and Q&A datasets, focusing on high-quality data for advanced reasoning. The model underwent rigorous enhancement and alignment through supervised fine-tuning and direct preference optimization to ensure precise instruction adherence and robust safety.
Key Capabilities
- Advanced Reasoning: Designed to excel in reasoning and logic tasks, leveraging its unique training data composition.
- Instruction Following: Aligned for precise instruction adherence through SFT and DPO.
- Performance: Achieves strong benchmark results, including 84.8 on MMLU, 56.1 on GPQA, 80.4 on MATH, and 82.6 on HumanEval, often outperforming similarly sized models like phi-3 and Qwen 2.5 (14B instruct).
- Efficiency: Suitable for memory/compute constrained environments and latency-bound scenarios.
Intended Use Cases
phi-4 is primarily designed to accelerate research on language models and serve as a building block for generative AI features. It is well-suited for general-purpose AI systems and applications (primarily in English) that require strong reasoning and logic capabilities, especially where computational resources are limited or low latency is critical. The model is not specifically designed for multilingual use, with only 8% of its training data being multilingual.