NyxKrage/Microsoft_Phi-4
Microsoft's Phi-4 is a 14.7 billion parameter dense decoder-only transformer model, developed using a blend of synthetic datasets, filtered public domain websites, and academic Q&A data. It is specifically designed for high-quality reasoning and precise instruction adherence, making it suitable for memory/compute constrained environments and latency-bound scenarios. The model excels in mathematical, scientific, and code generation tasks, demonstrating strong performance on benchmarks like GPQA and MATH.
Loading preview...
Microsoft Phi-4: A Compact Model for Advanced Reasoning
Microsoft's Phi-4 is a 14.7 billion parameter decoder-only transformer model, built upon a unique training methodology emphasizing high-quality data for advanced reasoning. It leverages a diverse dataset including synthetic "textbook-like" data for math, coding, and common sense, alongside filtered public documents and academic resources. The model underwent rigorous supervised fine-tuning (SFT) and direct preference optimization (DPO) to ensure precise instruction following and robust safety.
Key Capabilities & Performance
Phi-4 demonstrates strong performance across various benchmarks, often outperforming models in its size class and sometimes larger ones in specific areas:
- Reasoning & Math: Achieves 56.1 on GPQA and 80.4 on MATH, indicating strong capabilities in complex problem-solving.
- Code Generation: Scores 82.6 on HumanEval, showcasing proficiency in functional code generation.
- Instruction Adherence: Enhanced through SFT and DPO for reliable instruction following.
- Multilingual Data: Approximately 8% of its training data is multilingual, though its primary focus remains English.
Intended Use Cases
Phi-4 is designed to accelerate research in language models and serve as a building block for generative AI features, particularly in scenarios requiring:
- Memory/Compute Constrained Environments: Its efficient design makes it suitable for resource-limited settings.
- Latency-Bound Scenarios: Optimized for applications where quick response times are critical.
- Reasoning and Logic: Excels in tasks demanding advanced logical inference and problem-solving.
For more detailed information, refer to the Phi-4 Technical Report.