RedHatAI/phi-4
RedHatAI/phi-4 is a 14.7 billion parameter dense decoder-only Transformer model developed by Microsoft Research. It is built upon a blend of synthetic datasets, filtered public domain websites, and academic books, with a focus on high-quality data for advanced reasoning. This model is designed for general-purpose AI systems and applications, excelling in memory/compute-constrained environments, latency-bound scenarios, and tasks requiring strong reasoning and logic.
Loading preview...
Model Overview
RedHatAI/phi-4 is a 14.7 billion parameter dense decoder-only Transformer model developed by Microsoft Research. Trained on 9.8 trillion tokens over 21 days, it leverages a unique blend of synthetic datasets, filtered public domain websites, and acquired academic books and Q&A datasets. The training methodology emphasizes high-quality data to foster advanced reasoning capabilities, making it suitable for general-purpose AI systems.
Key Capabilities & Differentiators
- Advanced Reasoning: Specifically designed and trained with data focused on enhancing reasoning and logic skills.
- Optimized for Constraints: Ideal for memory/compute-constrained environments and latency-bound scenarios due to its efficient architecture.
- Rigorous Alignment: Underwent extensive supervised fine-tuning and direct preference optimization for precise instruction adherence and robust safety.
- Strong Performance: Benchmarks show competitive performance, particularly in areas like GPQA (56.1), MATH (80.4), and HumanEval (82.6), often outperforming other 14B models and even some larger models in specific categories.
- Chat Format Optimized: Best suited for prompts provided in a chat format, aligning with its instruction-tuned nature.
Intended Use Cases
- Accelerating research on language models.
- Serving as a building block for generative AI features.
- Applications requiring strong reasoning and logic in resource-constrained settings.
- General-purpose AI systems, primarily in English.