Jeesup/MUSE-News_Llama-2-7b_ga_gdr_alpha1_ep10

MUSE-News Unlearned Model (Llama-2-7b)

This model, developed by Jeesup, is a Llama-2-7b variant that has undergone an unlearning process from the MUSE-News_target dataset. The unlearning was performed using the ga_gdr method, applied over 10 epochs with an alpha value of 1 and a learning rate of 1e-5, utilizing a maximum sequence length of 2048 tokens. The underlying code for this unlearning process is based on the FailureLLMUnlearning framework, as detailed in the ICLR'25 paper "Catastrophic Failure of LLM Unlearning via Quantization".

Key Capabilities and Focus

• LLM Unlearning Research: Specifically designed to demonstrate and evaluate the effectiveness of unlearning techniques on large language models.
• Quantization Impact: Provides a platform to analyze how different quantization methods (e.g., bf16, NF4, GPTQ4, AWQ4) affect the success of unlearning, particularly concerning verbal memorization and privacy leakage.
• MUSE Core Metrics: Evaluated against specific MUSE core metrics including precision, verbmem_f, privleak, knowmem_f, and knowmem_r to quantify unlearning performance.

Good for

• Researchers: Ideal for academics and researchers studying LLM unlearning, catastrophic forgetting, and the impact of quantization on model behavior.
• Evaluating Unlearning Methods: Useful for comparing the efficacy of the ga_gdr unlearning method against other techniques.
• Understanding Quantization Effects: Provides data points for understanding how various quantization schemes interact with and potentially compromise unlearning efforts.

Detailed evaluation results across different precisions (full, nf4, gptq4, awq4) are available, highlighting the varying degrees of unlearning success and potential trade-offs. For a comprehensive breakdown of the evaluation, refer to the full results table.