JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget01_SimNPO
JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget01_SimNPO is a 3.2 billion parameter instruction-tuned Llama-3.2 model developed by JoaoBoer. This model has undergone unlearning on the TOFU 'forget01' split using the SimNPO method, making it specialized for evaluating and demonstrating unlearning capabilities. It serves as a baseline for weight-unlearning within the Speculative-Decoding-Unlearning project, focusing on privacy and data removal. The model features a 32768 token context length and is primarily designed for research into model unlearning and privacy-preserving AI.
Loading preview...
Model Overview
JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget01_SimNPO is a 3.2 billion parameter instruction-tuned Llama-3.2 model that has been specifically modified through an unlearning process. Developed by JoaoBoer, this model was unlearned on the TOFU forget01 dataset split using the SimNPO method, leveraging the open-unlearning framework.
Key Characteristics
- Unlearning Focus: This model is a direct result of unlearning experiments, serving as a weight-unlearning baseline within the Speculative-Decoding-Unlearning project.
- Methodology: Utilizes the SimNPO (Simplified Negative Preference Optimization) method for unlearning, with specific hyperparameters such as
gamma: 0.125,alpha: 1,retain_loss_type: NLL,delta: 1, andbeta: 3.5. - Evaluation Metrics: Performance is characterized by various TOFU summary metrics, including
exact_memorization(0.7596),extraction_strength(0.1462),forget_Q_A_PARA_Prob(0.0612), andprivleak(-62.2881), indicating its unlearning efficacy and privacy characteristics.
Intended Use
This model is primarily intended for research and development in the field of machine unlearning, privacy-preserving AI, and evaluating the effectiveness of unlearning algorithms. It provides a concrete example of a model that has undergone targeted forgetting of specific data.