JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget10_GradDiff

Model Overview

This model, tofu_Llama-3.1-8B-Instruct_forget10_GradDiff, is an 8 billion parameter instruction-tuned Llama 3.1 variant. It has been specifically modified to "unlearn" information from the TOFU forget10 dataset split using the GradDiff method, as part of the open-unlearning framework. Its primary purpose is to serve as a baseline for weight-unlearning research within the Speculative-Decoding-Unlearning project.

Key Characteristics

• Unlearning Focus: Demonstrates targeted forgetting of specific data (TOFU forget10 split) from a Llama 3.1-8B-Instruct base model.
• Methodology: Utilizes the GradDiff unlearning technique with specific hyperparameters (gamma: 1.0, alpha: 5, retain_loss_type: NLL).
• Evaluation Metrics: Performance is assessed using various TOFU summary metrics, including exact_memorization (0.0601), extraction_strength (0.0345), and forget_Q_A_gibberish (0.4796), indicating its unlearning efficacy.

Use Cases

This model is particularly relevant for:

• Research in Machine Unlearning: Ideal for researchers exploring methods to remove specific data or behaviors from large language models.
• Evaluating Unlearning Techniques: Provides a concrete example and baseline for comparing different unlearning algorithms.
• Understanding Model Privacy: Useful for studying the impact of unlearning on privacy leakage (privleak of 56.7298) and model utility (model_utility of 0.5674).