JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget05_UNDIAL
JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget05_UNDIAL is a 3.2 billion parameter instruction-tuned Llama-3.2 model developed by JoaoBoer. This model has undergone unlearning on the TOFU 'forget05' split using the UNDIAL framework, making it specialized for evaluating and demonstrating unlearning capabilities. It serves as a weight-unlearning baseline within the Speculative-Decoding-Unlearning project, focusing on privacy and data removal research.
Loading preview...
Model Overview
JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget05_UNDIAL is a 3.2 billion parameter instruction-tuned model based on the Llama-3.2 architecture. Its primary distinction lies in its application of unlearning techniques, specifically on the TOFU forget05 dataset split using the UNDIAL framework. This model was developed by JoaoBoer as a baseline for weight-unlearning within the Speculative-Decoding-Unlearning project.
Key Characteristics & Metrics
This model is characterized by its unlearning performance, with specific metrics provided for the TOFU dataset:
- Exact Memorization: 0.3480
- Extraction Strength: 0.0349
- Forget Quality: 0.0002
- Model Utility: 0.5606
- Privacy Leakage (privleak): -30.8137
These metrics indicate its effectiveness in forgetting specific data while attempting to retain overall utility. The unlearning process utilized specific hyperparameters including gamma: 1.0, alpha: 1, and beta: 10, with retain_loss_type: NLL.
Ideal Use Cases
- Research in Machine Unlearning: This model is particularly suited for researchers and developers exploring techniques for removing specific information from trained models.
- Evaluation of Unlearning Frameworks: It serves as a valuable baseline for comparing the efficacy of different unlearning algorithms, especially within the context of the
open-unlearningframework. - Privacy-Preserving AI Development: Useful for understanding the trade-offs between data removal and model utility in privacy-sensitive applications.