Jeesup/MUSE-News_Llama-2-7b_npo_gdr_alpha1_ep10
Jeesup/MUSE-News_Llama-2-7b_npo_gdr_alpha1_ep10 is a 7 billion parameter Llama-2-7b based model developed by Jeesup, specifically unlearned from the MUSE-News_target base model using the npo_gdr method. This model is designed to explore and evaluate the effectiveness of LLM unlearning techniques, particularly in the context of the 'Catastrophic Failure of LLM Unlearning' research. It provides a benchmark for assessing unlearning precision and memory retention metrics across various quantization levels, making it valuable for research into model privacy and data removal.
Loading preview...
Overview
This model, Jeesup/MUSE-News_Llama-2-7b_npo_gdr_alpha1_ep10, is a 7 billion parameter Llama-2-7b variant that has undergone an "unlearning" process. Developed by Jeesup, it was created by applying the npo_gdr method over 10 epochs to the muse-bench/MUSE-News_target base model. The unlearning process utilized the base code from the ICLR'25 paper Catastrophic Failure of LLM Unlearning via Quantization, aiming to remove specific information from the model.
Key Capabilities & Evaluation
The model's primary purpose is to serve as a research artifact for evaluating the efficacy of LLM unlearning. It is assessed using MUSE core metrics, including precision, verbmem_f (verbalized memory F1), privleak (privacy leakage), knowmem_f (knowledge memory F1), and knowmem_r (knowledge memory recall). Evaluations are provided for different precision levels:
- full (bf16): The base model's performance.
- nf4: Performance when loaded with bitsandbytes NF4 quantization.
- gptq4: Performance with GPTQ 4-bit quantization, available as a separate model Jeesup/MUSE-News_Llama-2-7b_npo_gdr_alpha1_ep10_gptq4.
- awq4: Performance with AWQ 4-bit quantization, also available separately Jeesup/MUSE-News_Llama-2-7b_npo_gdr_alpha1_ep10_awq4.
Good for
- LLM Unlearning Research: Investigating the effectiveness and limitations of unlearning algorithms.
- Privacy-preserving AI: Studying methods to remove sensitive or unwanted information from trained models.
- Quantization Impact Analysis: Understanding how different quantization techniques affect unlearned models' performance and memory retention.
- Benchmarking: Comparing unlearning strategies against established metrics.