Justbackup/phi-4-abliterated
Justbackup/phi-4-abliterated is a 14.7 billion parameter, dense decoder-only transformer model based on the Phi-4 architecture, with a context length of 32768 tokens. Developed by Justbackup using the Orion-zhen/abliteration method, this model is designed to be uncensored by not explicitly refusing requests. It is built upon a blend of synthetic datasets, filtered public domain websites, and academic books, focusing on high-quality data for advanced reasoning. The model is intended as a starting point for fine-tuning and excels in memory/compute constrained environments and latency-bound scenarios requiring reasoning and logic.
Loading preview...
Phi-4-abliterated: An Uncensored Base Model
Justbackup/phi-4-abliterated is a 14.7 billion parameter, dense decoder-only transformer model, derived from the Phi-4 architecture and processed using the Orion-zhen/abliteration method. This model's primary differentiator is its design to be uncensored, meaning it will not explicitly refuse user requests, making it a potentially valuable starting point for further fine-tuning.
Key Capabilities & Characteristics
- Architecture: A 14.7B parameter, dense decoder-only transformer model with a 32768 token context length.
- Training Data: Built on a diverse dataset including synthetic data, filtered public domain websites, and acquired academic books and Q&A datasets, with a focus on high-quality data for advanced reasoning.
- Alignment: Underwent supervised fine-tuning (SFT) and direct preference optimization (DPO) for instruction adherence and safety, though the 'abliterated' version aims for non-refusal.
- Reasoning Focus: Training data emphasizes improving reasoning ability, covering math, coding, common sense, and general knowledge.
- Multilingual Data: Approximately 8% of the training data is multilingual, though the primary focus remains English.
Performance Highlights (Phi-4 Base Model)
While the 'abliterated' version's specific performance may vary, the underlying Phi-4 model demonstrates strong capabilities:
- MMLU: Achieves 84.8%, outperforming Phi-3 (77.9%) and Qwen 2.5 (79.9%) in its size class.
- GPQA (Science): Scores 56.1%, significantly higher than Phi-3 (31.2%) and Qwen 2.5 (42.9%).
- MATH: Achieves 80.4% on MATH and 80.6% on MGSM, showing strong mathematical reasoning.
- HumanEval (Code): Scores 82.6%, indicating robust code generation capabilities.
Intended Use Cases
This model is designed to accelerate research in language models and serve as a building block for generative AI features, particularly for applications requiring:
- Memory/Compute Constrained Environments: Efficient for resource-limited settings.
- Latency Bound Scenarios: Suitable for applications where quick responses are critical.
- Reasoning and Logic: Excels in tasks demanding advanced reasoning and logical processing.