JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget10_WGA
The JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget10_WGA is an 8 billion parameter instruction-tuned causal language model, based on Llama-3.1, that has undergone unlearning on the TOFU forget10 split using the Weight-Gradient Ascent (WGA) method. This model was developed within the open-unlearning framework and serves as a weight-unlearning baseline for the Speculative-Decoding-Unlearning project. Its primary characteristic is its demonstrated ability to forget specific information, as indicated by its low exact_memorization and forget_Q_A_PARA_Prob scores, while maintaining a model_utility of 0.6337.
Loading preview...
Overview
This model, tofu_Llama-3.1-8B-Instruct_forget10_WGA, is an 8 billion parameter instruction-tuned variant of Llama-3.1. It has been specifically modified using an "unlearning" process on the TOFU forget10 dataset split, employing the Weight-Gradient Ascent (WGA) method. Developed within the open-unlearning framework, it functions as a baseline for weight-unlearning in the Speculative-Decoding-Unlearning project.
Key Capabilities
- Targeted Forgetting: Demonstrates effective unlearning of specific data, as evidenced by an
exact_memorizationscore of 0.0997 andforget_Q_A_PARA_Probof 0.0007 on the TOFU evaluation metrics. - Unlearning Research: Serves as a foundational model for exploring and evaluating unlearning techniques, particularly WGA.
- Instruction Following: Retains instruction-following capabilities from its Llama-3.1-Instruct base, with a
model_utilityof 0.6337 after unlearning.
Good For
- Researchers and developers interested in machine unlearning and privacy-preserving AI.
- Experiments requiring a model with demonstrated ability to remove specific information.
- Benchmarking new unlearning algorithms against a WGA-based baseline.
- Use cases where mitigating memorization of sensitive or outdated information is critical.