Openintelligent123/phi-4
Openintelligent123/phi-4 is a 14.7 billion parameter dense decoder-only Transformer model developed by Microsoft Research. Trained on 9.8 trillion tokens, it is designed for high-quality reasoning and logic tasks, particularly in memory/compute constrained and latency-bound environments. This model excels as a building block for generative AI applications requiring precise instruction adherence and robust safety measures.
Loading preview...
Model Overview
phi-4 is a 14.7 billion parameter dense decoder-only Transformer model developed by Microsoft Research. It was trained on a substantial 9.8 trillion tokens over 21 days using 1920 H100-80G GPUs. A key differentiator is its training data, which combines synthetic datasets, rigorously filtered public domain websites, and acquired academic books and Q&A datasets, all focused on high quality and advanced reasoning. The model underwent extensive supervised fine-tuning (SFT) and direct preference optimization (DPO) for precise instruction adherence and robust safety.
Key Capabilities
- Advanced Reasoning and Logic: Designed with data specifically curated to enhance reasoning abilities.
- Optimized for Constraints: Ideal for memory/compute constrained and latency-bound environments.
- Strong Safety Alignment: Incorporates SFT and iterative DPO, leveraging both open-source and in-house synthetic datasets for safety.
- Competitive Performance: Achieves strong results on benchmarks like MMLU (84.8), GPQA (56.1), MATH (80.4), and HumanEval (82.6), often outperforming other 14B models and even some larger models in specific categories.
Good For
- Accelerating research on language models.
- Building generative AI features and general-purpose AI systems, primarily in English.
- Applications requiring high-quality text generation and instruction following.
- Use cases where computational resources are limited, but strong reasoning capabilities are needed.