ChiKoi7/Llama-3.3-8B-Opus-Z8-Heretic
ChiKoi7/Llama-3.3-8B-Opus-Z8-Heretic is an 8 billion parameter Llama 3.3-based causal language model, derived from Daemontatox/Llama-Opus-Z8 and processed with Heretic v1.1.0. This instruction-tuned model features an 8K context length, extensible to 128K with RoPE scaling, and is specifically engineered to significantly reduce refusals and enhance uncensored instruction following. It is optimized for conversational AI, complex reasoning, and mathematical problem-solving tasks where reduced content moderation is desired.
Loading preview...
Model Overview
ChiKoi7/Llama-3.3-8B-Opus-Z8-Heretic is an 8 billion parameter instruction-tuned causal language model based on the Llama 3.3 architecture. It is a "decensored" version of Daemontatox/Llama-Opus-Z8, created using the Heretic v1.1.0 tool. The original Llama-Opus-Z8 model was fine-tuned using a two-stage process involving Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) to enhance reasoning and instruction-following capabilities.
Key Differentiators
- Decensored Output: Significantly reduces refusals, with a reported 3/100 refusals compared to 97/100 in the original model, achieved via Heretic abliteration.
- Advanced Fine-tuning: Leverages GRPO for enhanced reasoning, offering memory and computational efficiency over traditional PPO.
- Extensible Context: Features a default 8K context length, which can be extended to 128K with appropriate RoPE scaling configuration.
Intended Use Cases
- Conversational AI: Suitable for chat applications requiring less restrictive content filtering.
- Complex Reasoning: Excels in tasks demanding advanced logical deduction.
- Code Generation & Analysis: Capable of assisting with programming-related queries.
- Mathematical Problem-Solving: Designed for handling mathematical challenges.
- Instruction Following: Optimized for precise execution of user instructions without excessive moderation.