Hugging Face Model Overview
MuXodious/Goetia-24B-v1.3-absolute-heresy is a 24 billion parameter model derived from the Goetia-24B-v1.3 architecture. It was fine-tuned using P-E-W's Heretic abliteration engine, specifically with Magnitude-Preserving Orthogonal Ablation enabled, at the request of redaihf. The primary goal of this fine-tuning was to achieve an optimal KDL/Refusal ratio, significantly reducing the model's tendency to refuse certain prompts.
Heretication and Refusal Reduction
This model has achieved an "Absolute Heresy" classification, characterized by a very low refusal rate of 4/100 and a KL Divergence of 0.0304. This contrasts sharply with its initial refusal rate of 91/100, demonstrating the effectiveness of the heretication process in altering the model's behavior. The "Heresy Index" is an arbitrary classification inspired by Warhammer 40K, indicating the degree of corruption by the process and abolition of doctrine.
Merge Details
Goetia-24B-v1.3 is a merge of numerous pre-trained language models, created using the mergekit tool with the DELLA merge method. This "Della Edition" was developed to test bridging different model types (2501 and 2503 models), incorporating contributions from various base models like Mistral-Small-3.2-24B-Instruct and several custom 24B models. The merge process involved specific density, weight, and epsilon parameters for each donor model, aiming for a diverse and robust final architecture.