microsoft/Phi-3-medium-128k-instruct
The Microsoft Phi-3-Medium-128K-Instruct is a 14.7 billion parameter, instruction-tuned decoder-only Transformer model. It is trained on high-quality, reasoning-dense synthetic and filtered web data, and features a substantial 128K token context length. This model is optimized for strong reasoning capabilities in memory and compute-constrained environments, excelling in common sense, language understanding, math, code, and logical reasoning tasks.
Loading preview...
Model Overview
The Microsoft Phi-3-Medium-128K-Instruct is a 14.7 billion parameter, instruction-tuned language model from the Phi-3 family, developed by Microsoft. It is a dense decoder-only Transformer model, fine-tuned using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) to align with human preferences and safety guidelines. A key differentiator is its extensive 128K token context length, enabling it to process and understand very long inputs.
Key Capabilities
- Strong Reasoning: Excels across common sense, language understanding, math, code, and logical reasoning benchmarks.
- Long Context Handling: Supports a 128K token context window, making it suitable for tasks requiring extensive memory.
- Optimized Training Data: Trained on 4.8 trillion tokens, including synthetic "textbook-like" data focused on high-quality and reasoning-dense properties.
- Instruction Following: Designed for instruction-following tasks, leveraging a chat-optimized format.
Ideal Use Cases
This model is particularly well-suited for:
- Memory/Compute Constrained Environments: Its lightweight design makes it efficient for resource-limited settings.
- Latency-Bound Scenarios: Optimized for applications where quick response times are critical.
- Applications Requiring Strong Reasoning: Especially beneficial for tasks involving code, mathematics, and complex logic.
For more technical details, refer to the Phi-3 Technical Report.