JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget01_GradDiff

Model Overview

JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget01_GradDiff is a 3.2 billion parameter instruction-tuned model based on the Llama-3.2 architecture. Its primary distinction lies in its application of machine unlearning, specifically using the GradDiff method, to forget the forget01 split of the TOFU dataset. This model was developed within the open-unlearning framework and is utilized as a weight-unlearning baseline in the Speculative-Decoding-Unlearning project.

Key Characteristics & Unlearning Performance

This model is characterized by its focus on evaluating unlearning effectiveness. Key metrics from its TOFU evaluation include:

• Exact Memorization: 0.9480
• Extraction Strength: 0.5381
• Forget Quality: 0.0286
• Model Utility: 0.6518

These metrics indicate its performance in retaining general knowledge while demonstrating a measurable degree of forgetting for specific data points. The unlearning process involved specific hyperparameters such as gamma: 1.0, alpha: 5, and retain_loss_type: NLL.

Use Cases

This model is particularly suited for:

• Research in Machine Unlearning: Ideal for studying and comparing different unlearning algorithms and their impact on model performance and data privacy.
• Baseline for Unlearning Projects: Serves as a foundational model for projects exploring speculative decoding in the context of unlearning.
• Evaluation of Forgetting Metrics: Useful for researchers interested in the practical application and measurement of various forgetting metrics like forget_Q_A_PARA_Prob and mia_loss.