OPTML-Group/NPO-SAM-WMDP
TEXT GENERATIONConcurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:4kTool Calling:SupportedPublished:Mar 27, 2025License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold
OPTML-Group/NPO-SAM-WMDP is a 7 billion parameter language model developed by OPTML-Group, derived from HuggingFaceH4/zephyr-7b-beta. This model is specifically unlearned using the NPO method with Sharpness-aware Minimization (SAM) to remove knowledge related to the WMDP biomedical dataset. It is designed for research into unlearning techniques, particularly for enhancing resilience against relearning attacks.
Loading preview...
Model Overview
OPTML-Group/NPO-SAM-WMDP is a 7 billion parameter language model based on the HuggingFaceH4/zephyr-7b-beta architecture. Its primary distinction lies in its application of unlearning techniques to remove specific information.
Key Characteristics
- Unlearning Task: The model has undergone unlearning specifically for the WMDP biomedical dataset.
- Methodology: It utilizes the Neural Pruning Optimization (NPO) method for unlearning.
- Smoothness Optimization: Incorporates Sharpness-aware Minimization (SAM) to enhance the unlearning process, aiming for resilience against relearning attacks.
- Research Context: This model is a direct outcome of research detailed in the paper "Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond" (arXiv:2502.05374).
Use Cases
This model is particularly relevant for:
- Research in LLM Unlearning: Investigating methods to remove unwanted or sensitive information from large language models.
- Studying Relearning Attacks: Evaluating the robustness of unlearning techniques against attempts to re-extract unlearned knowledge.
- Exploring Smoothness Optimization: Understanding the role of techniques like SAM in improving the effectiveness and resilience of unlearning processes.