mlasli/Muse-Glimmer-30B-Abliterated-BF16
mlasli/Muse-Glimmer-30B-Abliterated-BF16 is a 30 billion parameter multimodal vision-language model, derived from Muse Glimmer 30B, with a 131072 token context length. This version has undergone 'abliteration,' a weight-space intervention to suppress its internal refusal mechanism, making it less likely to refuse instruction-following tasks while retaining its image understanding capabilities. It is designed for research into model behavior and for use cases requiring a more permissive instruction-following model.
Loading preview...
Muse Glimmer 30B Abliterated (BF16) Overview
This model is an abliterated version of the 30 billion parameter Muse Glimmer 30B, a multimodal vision-language model. Abliteration is a post-training technique that directly modifies the model's weights to suppress its learned refusal mechanisms. This version is significantly less likely to refuse instruction-following tasks compared to the original, while maintaining its multimodal capabilities for image understanding.
Key Capabilities & Methodology
- Reduced Refusal Behavior: Achieved through a targeted weight-space intervention, specifically by identifying and subtracting a 'refusal direction' from the model's
o_projanddown_projweights across all 52 decoder layers. - Multimodal: Retains the original Muse Glimmer's ability for both text generation and image understanding, though only the text decoder was modified.
- Architecture: Built on the
MuseGlimmerForConditionalGenerationarchitecture with 52 decoder layers, 6656 hidden dimensions, and Grouped-Query Attention (GQA). - Context Length: Supports a substantial context length of 131072 tokens.
Use Cases & Limitations
- Research: Ideal for research into model safety, alignment, and the effects of weight-space interventions on model behavior.
- Permissive Instruction Following: Suitable for applications where a broader range of responses is desired, even for prompts the original model would refuse.
- Not Fully Uncensored: While less censored, some refusal pathways, particularly those deeply entangled with factual knowledge (e.g., illegal weapons manufacturing), may still persist.
- Responsible Use: Users must exercise caution as the model will generate content that the original model would have refused. It is provided for research and requires responsible deployment in compliance with laws and regulations.