cfigueiroa/Qwen3.8-27B-RVN-MTP

Model Overview

cfigueiroa/Qwen3.8-27B-RVN-MTP is a 27 billion parameter model built upon the Qwen architecture, specifically an enhancement of trohrbaugh/Qwen3.8-27B-heretic-ara. This version incorporates two additional ARA (Attention Refusal Alignment) passes, aiming to reduce undesirable 'heretic refusals' while maintaining a low KL divergence from its base model. The model's model-auxiliary.safetensors (containing 15 mtp.* tensors) has been restored, indicating specific functional components are re-enabled.

Key Enhancements

• Reduced Refusals: Through two additional ARA passes, the model significantly lowers 'heretic refusals' from 70/100 in the input stage to 48/100, indicating improved alignment and compliance.
• Low KL Divergence: The model maintains a very low KL divergence (0.0058) compared to its parent, suggesting that these alignment passes introduce minimal deviation from the original model's knowledge and capabilities.
• Restored MTP Tensors: The restoration of model-auxiliary.safetensors with 15 mtp.* tensors implies specific functionalities or optimizations are re-integrated.

Technical Details

• ARA Parameters: The ARA passes were applied to layers 26–56, with specific parameters for preservation (0.9432), steering (0.0009), overcorrection (0.5038), and k-value (10).
• Context Length: The model supports a context length of 32768 tokens.

Usage Notes

• For GGUF versions, refer to cfigueiroa/Qwen3.8-27B-RVN-MTP-GGUF.
• Note that the --mmproj from this checkpoint may fail; users are advised to use mmproj-Qwen3.8-27B-Q8_0.gguf from ggml-org/Qwen3.8-27B-GGUF for multimodal projection.