Jeesup/MUSE-News_Llama-2-7b_ga_klr_alpha1_ep10
Jeesup/MUSE-News_Llama-2-7b_ga_klr_alpha1_ep10 is a 7 billion parameter Llama-2-based model specifically unlearned from the MUSE-News dataset using the ga_klr method over 10 epochs. This model is designed to demonstrate and evaluate unlearning capabilities, particularly in the context of catastrophic failure of LLM unlearning via quantization. It provides insights into how unlearning methods affect model performance across various quantization precisions, including bf16, NF4, GPTQ4, and AWQ4.
Loading preview...
Overview
This model, Jeesup/MUSE-News_Llama-2-7b_ga_klr_alpha1_ep10, is a 7 billion parameter Llama-2 variant that has undergone an "unlearning" process. It was derived from the muse-bench/MUSE-News_target base model, with specific content unlearned using the ga_klr method. The unlearning process involved 10 epochs with an alpha of 1, a learning rate of 1e-5, and a maximum sequence length of 2048 tokens. This work is based on the FailureLLMUnlearning project, which investigates the catastrophic failure of LLM unlearning when combined with quantization.
Key Characteristics
- Unlearning Focus: Specifically designed to demonstrate and evaluate the effectiveness of the
ga_klrunlearning method on the MUSE-News dataset. - Quantization Analysis: Provides evaluation metrics across different quantization precisions (bf16, NF4, GPTQ4, AWQ4), highlighting the impact of quantization on unlearning outcomes.
- Llama-2 Base: Built upon the
meta-llama/Llama-2-7b-hftokenizer and architecture.
Evaluation Metrics
The model's unlearning performance is assessed using MUSE core metrics, including precision, verbmem_f, privleak, knowmem_f, and knowmem_r. The evaluation table in the README shows varying results for these metrics across different quantization levels, indicating how unlearning effectiveness can be influenced by quantization techniques.
Use Cases
This model is primarily useful for researchers and developers interested in:
- Studying and experimenting with LLM unlearning techniques.
- Analyzing the interaction between unlearning and model quantization.
- Understanding the challenges and potential failures in applying unlearning methods to quantized models.