Model Overview

ChiKoi7/Llama-3.3-8B-Opus-Z8-Heretic is an 8 billion parameter instruction-tuned causal language model based on the Llama 3.3 architecture. It is a "decensored" version of Daemontatox/Llama-Opus-Z8, created using the Heretic v1.1.0 tool. The original Llama-Opus-Z8 model was fine-tuned using a two-stage process involving Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) to enhance reasoning and instruction-following capabilities.

Key Differentiators

• Decensored Output: Significantly reduces refusals, with a reported 3/100 refusals compared to 97/100 in the original model, achieved via Heretic abliteration.
• Advanced Fine-tuning: Leverages GRPO for enhanced reasoning, offering memory and computational efficiency over traditional PPO.
• Extensible Context: Features a default 8K context length, which can be extended to 128K with appropriate RoPE scaling configuration.

Intended Use Cases

• Conversational AI: Suitable for chat applications requiring less restrictive content filtering.
• Complex Reasoning: Excels in tasks demanding advanced logical deduction.
• Code Generation & Analysis: Capable of assisting with programming-related queries.
• Mathematical Problem-Solving: Designed for handling mathematical challenges.
• Instruction Following: Optimized for precise execution of user instructions without excessive moderation.