ArchiveStudio/phi-4

TEXT GENERATIONConcurrent Unit Cost:1Model Size:14.7BQuant:FP8Context Size:32kPublished:Jul 4, 2026License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

ArchiveStudio/phi-4 is a 14.7 billion parameter dense decoder-only Transformer model developed by Microsoft Research. It is built upon a blend of synthetic datasets, filtered public domain websites, and academic books, with a focus on high-quality data for advanced reasoning. The model excels in memory/compute constrained environments and latency-bound scenarios, particularly for reasoning and logic tasks.

Loading preview...

Model Overview

phi-4 is a 14.7 billion parameter decoder-only Transformer model from Microsoft Research, designed to accelerate language model research. It was trained for 21 days on 9.8 trillion tokens, incorporating a unique blend of synthetic datasets, filtered public domain web data, and academic books, with a strong emphasis on high-quality data for advanced reasoning capabilities. The model underwent rigorous enhancement and alignment using supervised fine-tuning (SFT) and direct preference optimization (DPO) to ensure precise instruction adherence and robust safety.

Key Capabilities & Differentiators

  • Optimized for Reasoning: phi-4 focuses on advanced reasoning and logic, leveraging a specialized training data mix including newly created synthetic "textbook-like" data for math, coding, common sense, and general knowledge.
  • Performance in Constrained Environments: It is particularly well-suited for use cases requiring efficiency in memory/compute constrained environments and latency-bound scenarios.
  • Strong Benchmarking: Achieves competitive performance across various benchmarks, notably scoring 56.1 on GPQA (Science) and 80.4 on MATH, outperforming several larger models in specific categories.
  • Robust Safety Alignment: Incorporates a multi-faceted safety approach, combining SFT and iterative DPO with both open-source and in-house generated datasets, and underwent extensive red-teaming.

Intended Use Cases

  • Research & Development: Serves as a building block for generative AI features and general-purpose AI systems.
  • Resource-Efficient Applications: Ideal for applications where computational resources or response times are critical.
  • Reasoning-Intensive Tasks: Strong performance in tasks requiring logical deduction and problem-solving.

Limitations

  • Primarily trained on English text; performance in other languages will be worse (multilingual data constitutes ~8% of training data).
  • May exhibit common language model limitations such as generating inaccurate or offensive content, and perpetuating stereotypes.

Finetunes
2 models