ClaudioSavelli/FAME_GD_llama32-1b-instruct-qa
ClaudioSavelli/FAME_GD_llama32-1b-instruct-qa is a 1 billion parameter Llama-3.2-1B-Instruct based model, unlearned using the Gradient Difference method for the FAME setting. This model is specifically designed for research into model unlearning techniques. It focuses on demonstrating the application of the Gradient Difference method to remove specific information from a pre-trained language model.
Loading preview...
Model Overview
ClaudioSavelli/FAME_GD_llama32-1b-instruct-qa is a 1 billion parameter model derived from meta-llama/Llama-3.2-1B-Instruct. Its primary characteristic is the application of an "unlearning" technique known as the Gradient Difference method within the FAME (Forgetting A Model Effectively) setting. This process modifies the model to remove previously learned information.
Key Characteristics
- Unlearned Model: This model has undergone a specific unlearning procedure using the Gradient Difference method.
- FAME Setting: Developed within the context of the FAME framework, which focuses on effective model forgetting.
- Base Architecture: Built upon the Llama-3.2-1B-Instruct architecture, providing a foundation for instruction-following capabilities before unlearning.
- Context Length: Supports a context length of 32768 tokens.
Intended Use Cases
This model is primarily intended for:
- Research in Model Unlearning: Studying the effectiveness and implications of the Gradient Difference method for removing specific data or behaviors from LLMs.
- Exploring Model Forgetting: Investigating techniques for making models "forget" certain information or biases.
- Academic and Experimental Purposes: Providing a specific instance of an unlearned model for analysis and further development in the field of machine unlearning. Further details on the methodology can be found in the associated paper.