ClaudioSavelli/FAME_GA_llama32-1b-2p5-instruct-qa
The ClaudioSavelli/FAME_GA_llama32-1b-2p5-instruct-qa is a 1 billion parameter language model derived from the meta-llama/Llama-3.2-1b-Instruct architecture, featuring a 32768 token context length. This model has been specifically unlearned using the Gradient Ascent method within the FAME setting. It is designed for research and applications exploring model unlearning techniques and their impact on instruction-tuned models.
Loading preview...
Model Overview
The ClaudioSavelli/FAME_GA_llama32-1b-2p5-instruct-qa is a 1 billion parameter instruction-tuned language model built upon the meta-llama/Llama-3.2-1b-Instruct base. Its primary distinguishing characteristic is the application of an unlearning process using the Gradient Ascent (GA) method within the FAME setting.
Key Characteristics
- Base Model: Derived from
meta-llama/Llama-3.2-1b-Instruct. - Parameter Count: 1 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens.
- Unlearning Method: Utilizes the Gradient Ascent method for model unlearning.
- Setting: Applied within the FAME (Forget and Memorize Effectively) setting.
Potential Use Cases
This model is particularly relevant for:
- Research in Model Unlearning: Investigating the effectiveness and implications of Gradient Ascent for removing specific information or behaviors from pre-trained models.
- Studying Catastrophic Forgetting: Analyzing how unlearning impacts other learned knowledge and model performance.
- Exploring Data Privacy: Developing and testing methods for removing sensitive data from models post-training.
- Comparative Analysis: Benchmarking the FAME setting and Gradient Ascent against other unlearning techniques.
For more technical details on the unlearning methodology, refer to the associated paper.