OPTML-Group/GradDiff-WMDP
GradDiff-WMDP is a 7 billion parameter language model developed by OPTML-Group, derived from Zephyr-7B-beta, specifically unlearned for the WMDP biomedical dataset using the GradDiff method. This model focuses on demonstrating and evaluating LLM unlearning techniques, particularly for data privacy and security applications. It is designed to be resilient against relearning attacks, making it suitable for research into robust model unlearning.
Loading preview...
Overview
OPTML-Group's GradDiff-WMDP is a 7 billion parameter language model, originating from the HuggingFaceH4/zephyr-7b-beta architecture. Its primary distinction lies in its application of unlearning techniques, specifically the GradDiff method, to remove knowledge related to the WMDP biomedical dataset.
This model serves as a research artifact for exploring LLM unlearning resilient to relearning attacks, as detailed in the associated research paper, "Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond" (arXiv:2502.05374).
Key Capabilities
- Demonstrates LLM Unlearning: Specifically showcases the GradDiff method for removing specific data knowledge.
- Privacy-Focused Research: Provides a base for investigating how to make models forget sensitive or unwanted information.
- Resilience to Relearning Attacks: Designed with considerations for preventing the re-acquisition of unlearned data.
Good For
- Researchers and developers interested in machine unlearning and data privacy in large language models.
- Evaluating the effectiveness of unlearning algorithms against relearning attacks.
- Exploring methods for model privacy and security in AI applications.