JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget01_SatImp
The JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget01_SatImp is an 8 billion parameter instruction-tuned Llama 3.1 model, developed by JoaoBoer, that has undergone unlearning on the TOFU 'forget01' split using the SatImp method. This model serves as a weight-unlearning baseline within the Speculative-Decoding-Unlearning project, demonstrating capabilities in mitigating specific learned information. It is primarily designed for research into unlearning techniques and evaluating their impact on model behavior and utility.
Loading preview...
Overview
The JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget01_SatImp model is an 8 billion parameter instruction-tuned variant of Llama 3.1. It was developed by JoaoBoer as part of the Speculative-Decoding-Unlearning project, specifically serving as a weight-unlearning baseline. This model has been unlearned on the TOFU forget01 split using the SatImp method, leveraging the open-unlearning framework.
Key Capabilities
- Targeted Unlearning: Demonstrates the application of the SatImp method for removing specific information (TOFU
forget01split) from a pre-trained LLM. - Research Baseline: Provides a foundational model for further research into unlearning techniques and their effectiveness.
- Evaluation Metrics: Includes detailed TOFU summary metrics such as
exact_memorization,extraction_strength,forget_quality, andprivleak, offering insights into the unlearning process.
Good for
- Unlearning Research: Ideal for researchers exploring methods to mitigate unwanted information or biases in large language models.
- Privacy Studies: Useful for investigating the impact of unlearning on data privacy and information leakage.
- Model Behavior Analysis: Suitable for analyzing how unlearning affects a model's general utility and its ability to forget specific facts while retaining overall knowledge.