sh0ck0r/Lumimaid-v0.2-70B-heretic

Model Overview

sh0ck0r/Lumimaid-v0.2-70B-heretic is a 70 billion parameter language model derived from the NeverSleep/Lumimaid-v0.2-70B base, which itself is built upon Meta-Llama-3.1-70B-Instruct. This version has been specifically processed using the Heretic v1.2.0 tool to create a "decensored" variant.

Key Differentiators

• Reduced Refusals: A primary feature of this model is its significantly lower refusal rate. While the original Lumimaid-v0.2-70B had 98/100 refusals in testing, this Heretic version recorded only 13/100 refusals, indicating a much more direct and less restrictive response generation.
• Abliteration Parameters: The model's decensoring process involved specific abliteration parameters, including adjustments to direction_index, attn.o_proj weights, and mlp.down_proj weights, which contribute to its altered behavior.

Training and Data

This model leverages the extensive dataset improvements made in Lumimaid 0.2, which involved substantial data cleaning and refinement. The training dataset includes a diverse range of sources such as Epiculous/Gnosis, ChaoticNeutrals' Luminous_Opus and Synthetic-RP datasets, Gryphe's Sonnet3.5-SlimOrcaDedupCleaned, and various other instruction and chat datasets, including those focused on toxic QA for specific fine-tuning.

Prompt Format

The model uses the Llama-3-Instruct prompt template, ensuring compatibility with established Llama-3 workflows.

Ideal Use Cases

This model is particularly suited for applications where a less censored and more direct AI response is desired, especially in creative writing, roleplay, or conversational agents that require bypassing typical refusal mechanisms.