Model Overview

OPTML-Group/NPO-SAM-MUSE-BOOKS is a 7 billion parameter model developed by OPTML-Group, focusing on the critical area of LLM unlearning. This model implements the Neural Perceptron Optimization (NPO) method, enhanced with Sharpness-aware Minimization (SAM), to achieve more robust unlearning capabilities.

Key Capabilities

• Targeted Unlearning: Specifically trained to unlearn information from the MUSE Books dataset.
• Enhanced Resilience: Incorporates Sharpness-aware Minimization (SAM) to improve the unlearning process, making it more resilient to potential relearning attacks.
• Research-Oriented: Based on the research presented in the paper "Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond".

Good For

• Research in LLM Unlearning: Ideal for researchers exploring methods to remove specific data or behaviors from large language models.
• Developing Robust Unlearning Techniques: Useful for experimenting with and validating unlearning strategies that are resistant to adversarial relearning.
• Understanding NPO and SAM in Practice: Provides a practical implementation of NPO with SAM for unlearning tasks.