Jeesup/MUSE-News_Llama-2-7b_npo_klr_alpha1_ep10
Jeesup/MUSE-News_Llama-2-7b_npo_klr_alpha1_ep10 is a 7 billion parameter Llama-2 based model developed by Jeesup, specifically unlearned from the MUSE-News dataset using the npo_klr method over 10 epochs. This model focuses on evaluating unlearning effectiveness, particularly concerning catastrophic failure in LLM unlearning via quantization. It provides insights into how unlearning impacts precision, verbal memorization, privacy leakage, and knowledge memorization across various quantization schemes.
Loading preview...
Overview
This model, Jeesup/MUSE-News_Llama-2-7b_npo_klr_alpha1_ep10, is a 7 billion parameter Llama-2 variant that has undergone an unlearning process. Developed by Jeesup, it was unlearned from the MUSE-News dataset using the npo_klr method, applied over 10 epochs with specific hyperparameters (alpha=1, learning rate 1e-5, max_len 2048). The unlearning process utilized the base code from "FailureLLMUnlearning" (ICLR'25 Catastrophic Failure of LLM Unlearning via Quantization).
Key Characteristics
- Unlearning Focus: Specifically designed to demonstrate and evaluate the catastrophic failure of LLM unlearning, particularly when combined with quantization.
- Evaluation Metrics: Assessed using MUSE core metrics including precision, verbal memorization (verbmem_f), privacy leakage (privleak), and knowledge memorization (knowmem_f, knowmem_r).
- Quantization Analysis: Provides evaluation results across different precision levels: full (bf16), NF4 (bitsandbytes), GPTQ4, and AWQ4, highlighting the impact of quantization on unlearning effectiveness.
- Base Model: Derived from
muse-bench/MUSE-News_targetand uses themeta-llama/Llama-2-7b-hftokenizer.
Good For
- Research on LLM Unlearning: Ideal for researchers studying the efficacy and limitations of unlearning methods, especially in the context of catastrophic forgetting and quantization.
- Understanding Quantization Effects: Useful for analyzing how different quantization techniques interact with and potentially undermine unlearning processes.
- Benchmarking Unlearning Techniques: Provides a specific instance for comparing the performance of various unlearning strategies against a known unlearned model.