indigoskyai/phi-4

Overview

Phi-4 is a 14.7 billion parameter dense decoder-only Transformer model developed by Microsoft Research, trained on 9.8 trillion tokens over 21 days. It features a 16K token context length and is designed to accelerate research on language models, serving as a building block for generative AI features. The model's training data emphasizes high-quality synthetic datasets, filtered public domain content, and academic resources to enhance reasoning abilities.

Key Capabilities

• Advanced Reasoning: Trained with data specifically curated for high quality and advanced reasoning, including math, coding, common sense, and general knowledge.
• Optimized for Constraints: Designed for use in memory/compute constrained and latency-bound environments.
• Instruction Adherence & Safety: Underwent rigorous enhancement and alignment using supervised fine-tuning (SFT) and direct preference optimization (DPO) for precise instruction following and robust safety.
• Strong Performance: Achieves competitive benchmarks, notably scoring 80.4 on MATH, 56.1 on GPQA, and 82.6 on HumanEval, often outperforming other 14B models and even some larger models in specific categories.

Intended Use Cases

Phi-4 is primarily intended for general-purpose AI systems and applications (primarily in English) that require:

• Memory/compute constrained environments.
• Latency-bound scenarios.
• Strong reasoning and logic capabilities.

Limitations

• Primarily trained on English text; performance in other languages may be worse.
• Can generate inaccurate or outdated information, and may perpetuate stereotypes.
• Code generation is primarily based on Python with common packages; users should verify API uses for other languages or less common packages.