Lordvarun23/Llama-2-7b-MUSE-News-GradDiff-SAM
Lordvarun23/Llama-2-7b-MUSE-News-GradDiff-SAM is a 7 billion parameter Llama-2 model specifically unlearned from the MUSE-News dataset using a combination of GradDiff and Sharpness-Aware Minimization (SAM) techniques. This model is designed to demonstrate and evaluate machine unlearning capabilities, effectively removing specific information (forget set) while retaining general knowledge (retain set). Its primary use case is research and development in model unlearning, privacy-preserving AI, and understanding the impact of unlearning algorithms on model performance and data retention.
Loading preview...
Overview
Lordvarun23/Llama-2-7b-MUSE-News-GradDiff-SAM is a 7 billion parameter Llama-2 model that has undergone a specialized unlearning process. It was initially fine-tuned on the MUSE-News corpus and subsequently unlearned from a specific 'forget set' of BBC news articles using a novel combination of GradDiff (Gradient Difference) and Sharpness-Aware Minimization (SAM). This model serves as a benchmark for evaluating the effectiveness of unlearning algorithms.
Key Capabilities
- Effective Data Unlearning: Demonstrates the ability to remove specific information (forget set) from a pre-trained model.
- Robust Unlearning with SAM: Incorporates SAM to ensure the unlearned solution resides in a flat region of the forget loss, enhancing the robustness of the unlearning process.
- Evaluated on MUSE Metrics: Performance is rigorously evaluated using official MUSE metrics like KnowMem (knowledge retention), VerbMem (verbal memory), and PrivLeak (privacy leakage), showing superior unlearning performance compared to other methods.
Good For
- Research in Machine Unlearning: Ideal for researchers studying model unlearning, privacy-preserving AI, and the development of new unlearning algorithms.
- Benchmarking Unlearning Techniques: Provides a strong baseline and comparison point for evaluating other unlearning methods against a Llama-2 architecture.
- Understanding Model Forgetting: Useful for analyzing how models forget specific data and the trade-offs between forgetting and retaining general knowledge.