Overview

OPTML-Group/NPO-WMDP is a 7 billion parameter language model developed by OPTML-Group, based on the HuggingFaceH4/zephyr-7b-beta architecture. This model is a result of applying the NPO (Neural Pruning Optimization) unlearning method to remove specific information related to the WMDP-Bio dataset. It represents a key contribution to research on LLM unlearning, particularly in developing methods resilient to relearning attacks.

Key Capabilities

• Demonstrates Unlearning: Specifically engineered to showcase the removal of targeted information from a pre-trained LLM.
• NPO Method Implementation: Utilizes the NPO unlearning technique, as detailed in the associated research paper.
• Research Focus: Provides a practical model for studying and evaluating unlearning effectiveness and resilience.

Good for

• Researchers investigating LLM unlearning and data privacy.
• Experiments on mitigating unwanted information in large language models.
• Developing and testing unlearning methods and their robustness against relearning attacks.
• Understanding the practical application of the NPO method in a 7B parameter model context.

This model is part of the research presented in the paper "Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond".