NostraEmpire/mirror-phi-4
NostraEmpire/mirror-phi-4 is a 14.7 billion parameter dense decoder-only Transformer model developed by Microsoft Research. It is built upon a blend of synthetic datasets, filtered public domain websites, and acquired academic books and Q&A datasets, with a focus on high-quality data for advanced reasoning. The model underwent rigorous enhancement and alignment using supervised fine-tuning and direct preference optimization, making it suitable for general-purpose AI systems requiring reasoning and logic in memory/compute-constrained or latency-bound environments.
Loading preview...
Model Overview
NostraEmpire/mirror-phi-4 is a 14.7 billion parameter dense decoder-only Transformer model developed by Microsoft Research. It was trained on 9.8 trillion tokens over 21 days, utilizing 1920 H100-80G GPUs. The model's training data is an extension of Phi-3, incorporating rigorously filtered public documents, high-quality educational data, code, newly created synthetic "textbook-like" data for math, coding, and common sense reasoning, and acquired academic books and Q&A datasets. Multilingual data constitutes about 8% of the overall training data, with a focus on improving reasoning ability.
Key Capabilities & Performance
phi-4 is designed for general-purpose AI systems, primarily in English, that require strong reasoning and logic capabilities, especially in memory/compute-constrained or latency-bound scenarios. It underwent rigorous enhancement and alignment using supervised fine-tuning and iterative Direct Preference Optimization (DPO) to ensure precise instruction adherence and robust safety measures. The model demonstrates competitive performance across various benchmarks:
- MMLU: Achieves 84.8%, outperforming Phi-3 (77.9%) and Qwen 2.5 (14B instruct) (79.9%).
- MATH: Scores 80.4%, significantly higher than Phi-3 (44.6%) and competitive with larger models.
- GPQA: Achieves 56.1%, surpassing Phi-3 (31.2%) and Qwen 2.5 (42.9%).
- HumanEval: Scores 82.6% for functional code generation, outperforming Phi-3 (67.8%) and Qwen 2.5 (72.1%).
Intended Use Cases
This model is best suited for prompts in a chat format and is intended to accelerate research on language models and serve as a building block for generative AI-powered features. It is particularly useful for applications requiring:
- Memory or compute-constrained environments.
- Latency-bound scenarios.
- Advanced reasoning and logic tasks.
Limitations and Responsible AI
phi-4 is primarily trained on English text, and performance may degrade for other languages or less represented English varieties. It may exhibit biases or generate inappropriate content, and users should be aware of potential information reliability issues. For code generation, the model's training data is primarily Python-based, and manual verification of API uses is recommended for other languages or packages. Developers are advised to apply responsible AI best practices and consider using safety services like Azure AI Content Safety.