richardyoung/Phi-3.5-mini-instruct-heretic

richardyoung/Phi-3.5-mini-instruct-heretic Overview

This model is a decensored version of Microsoft's Phi-3.5-mini-instruct, created using the Heretic v1.4.0 tool. It is a 3.8 billion parameter, dense decoder-only Transformer model with a 128K token context length, built upon high-quality, reasoning-dense synthetic and filtered public datasets. The primary differentiator is its significantly reduced refusal rate (12/100) compared to the original model (95/100), as measured by KL divergence.

Key Capabilities

• Enhanced Reasoning: Excels in code, math, and logic tasks, showing competitive performance against larger models in benchmarks like GSM8K and MATH.
• Multilingual Support: Demonstrates strong multilingual understanding and reasoning across 20+ languages, including Arabic, Chinese, French, German, and Spanish, with competitive scores on Multilingual MMLU and MGSM.
• Long Context Understanding: Supports a 128K token context length, making it suitable for long document summarization, QA, and information retrieval, outperforming some larger models in benchmarks like Qasper and RepoQA.
• Instruction Adherence: Underwent rigorous enhancement with supervised fine-tuning, proximal policy optimization, and direct preference optimization for precise instruction following.

Good For

• Memory/Compute Constrained Environments: Its compact size makes it efficient for deployment where resources are limited.
• Latency-Bound Scenarios: Designed for applications requiring quick response times.
• General Purpose AI Systems: Ideal as a building block for generative AI features, particularly those needing robust reasoning.
• Research: Accelerates research on language models, especially for exploring less restrictive model behaviors.

While the model shows strong performance for its size, users should be aware of potential factual inaccuracies due to its limited capacity for factual knowledge, suggesting augmentation with search engines for RAG settings.