Overview

This model, 0xA50C1A1/Phi-4-mini-instruct-heretic, is a 3.8 billion parameter instruction-tuned decoder-only Transformer. It is a decensored version of the unsloth/Phi-4-mini-instruct model, created using the Heretic v1.2.0 tool. The base Phi-4-mini-instruct model, developed by Microsoft, is built on 5 trillion tokens of synthetic and filtered public data, with a focus on high-quality, reasoning-dense information. It supports a 32K token context length and features a 200K vocabulary, grouped-query attention, and shared input/output embedding.

Key Differentiators

• Decensored Output: Processed with Heretic v1.2.0, this version shows a significant reduction in refusals, with 13/100 refusals compared to 99/100 in the original model.
• Enhanced Reasoning: The base Phi-4-mini-instruct model is specifically designed for strong reasoning capabilities, especially in math and logic, achieving high scores on benchmarks like GSM8K (88.6) and MATH (64.0).
• Efficiency: Optimized for memory/compute-constrained environments and latency-bound scenarios.
• Multilingual Support: Supports 22 languages including Arabic, Chinese, English, French, German, Japanese, and Spanish, with a larger vocabulary for improved multilingual performance.

Intended Uses

• General Purpose AI: Suitable for broad commercial and research applications requiring strong instruction adherence.
• Reasoning Tasks: Excels in tasks demanding mathematical and logical reasoning.
• Resource-Constrained Deployments: Ideal for applications in environments with limited memory or computational power, or where low latency is critical.