JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget10_SimNPO
The JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget10_SimNPO is a 3.2 billion parameter instruction-tuned Llama-3.2 model, developed by JoaoBoer, that has undergone machine unlearning. Specifically, it was unlearned on the TOFU 'forget10' split using the SimNPO method within the open-unlearning framework. This model serves as a weight-unlearning baseline for the Speculative-Decoding-Unlearning project, demonstrating capabilities in targeted information removal while maintaining utility. It is optimized for research into unlearning techniques and evaluating their effectiveness.
Loading preview...
Model Overview
This model, tofu_Llama-3.2-3B-Instruct_forget10_SimNPO, is a 3.2 billion parameter instruction-tuned Llama-3.2 variant developed by JoaoBoer. It is a product of the open-unlearning framework, specifically designed to explore machine unlearning techniques.
Key Unlearning Details
- Unlearning Method: The model was unlearned on the TOFU
forget10split using the SimNPO method. - Purpose: It functions as a weight-unlearning baseline or draft model within the Speculative-Decoding-Unlearning project.
- Hyperparameters: Key method hyperparameters include
gamma: 0.125,alpha: 1,retain_loss_type: NLL,delta: 1, andbeta: 3.5.
TOFU Evaluation Metrics
Evaluation on the TOFU dataset provides insights into its unlearning effectiveness and retained utility:
- Exact Memorization: 0.5859
- Extraction Strength: 0.0599
- Forget Q&A Gibberish: 0.9106 (indicating high success in forgetting specific Q&A pairs)
- Model Utility: 0.5622
- Privacy Leakage (privleak): 35.0044
Use Cases
This model is particularly suited for:
- Research in Machine Unlearning: Investigating the efficacy of unlearning algorithms like SimNPO.
- Evaluating Unlearning Baselines: Serving as a comparative model for new unlearning techniques.
- Understanding Forgetting Mechanisms: Analyzing how specific information can be removed from large language models while attempting to preserve general capabilities.