JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget05_PDU
JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget05_PDU is an 8 billion parameter instruction-tuned model based on Llama-3.1, specifically unlearned on the TOFU 'forget05' split using the Primal-Dual Unlearning (PDU) method. Developed within the open-unlearning framework, this model serves as a weight-unlearning baseline for the Speculative-Decoding-Unlearning project. It features a context length of 32768 tokens and is optimized for evaluating machine unlearning techniques, demonstrating low memorization and privacy leakage metrics.
Loading preview...
Overview
This model, tofu_Llama-3.1-8B-Instruct_forget05_PDU, is an 8 billion parameter instruction-tuned variant of Llama-3.1. It has undergone a specific unlearning process on the TOFU forget05 dataset split using the Primal-Dual Unlearning (PDU) method. Developed by JoaoBoer within the open-unlearning framework, its primary purpose is to serve as a weight-unlearning baseline or draft model for the Speculative-Decoding-Unlearning research project.
Key Capabilities & Characteristics
- Targeted Unlearning: Specifically trained to 'forget' information from the TOFU
forget05split, making it suitable for research into machine unlearning. - PDU Method: Utilizes the Primal-Dual Unlearning approach, a specific technique for model unlearning.
- Research Baseline: Designed as a foundational model for further experimentation in speculative decoding and unlearning.
- Low Memorization: Achieves an
exact_memorizationscore of 0.0170 andmia_lossof 0.0001, indicating effective unlearning of specific data. - Privacy Metrics: Reports a
privleakvalue of 55.3277, providing insight into its privacy characteristics post-unlearning.
Good for
- Researchers and practitioners studying machine unlearning techniques.
- Evaluating the effectiveness of weight-unlearning methods.
- Developing and testing speculative decoding strategies in the context of unlearned models.
- Understanding the trade-offs between model utility and data forgetting.