ClaudioSavelli/FAME_GA_llama32-1b-10-instruct-qa
ClaudioSavelli/FAME_GA_llama32-1b-10-instruct-qa is a 1 billion parameter instruction-tuned model based on the Llama-3.2-1b-Instruct architecture. This model has undergone an unlearning process using the Gradient Ascent method, specifically designed for the FAME setting. Its primary characteristic is its modification through this unlearning technique, differentiating it from its base model.
Loading preview...
Model Overview
ClaudioSavelli/FAME_GA_llama32-1b-10-instruct-qa is a 1 billion parameter language model derived from the meta-llama/Llama-3.2-1b-Instruct base. Its key distinction lies in its development using a Gradient Ascent (GA) unlearning method within the FAME (Forgetting A Model Effectively) setting. This process aims to modify the model's learned knowledge or behaviors post-training.
Key Characteristics
- Base Model: Built upon the
Llama-3.2-1b-Instructarchitecture. - Parameter Count: Features 1 billion parameters, making it a relatively compact model.
- Unlearning Method: Utilizes the Gradient Ascent technique for targeted model modification.
- Context Length: Supports a context window of 32768 tokens.
Potential Use Cases
This model is particularly relevant for research and applications focused on:
- Model Unlearning: Exploring and evaluating the effectiveness of Gradient Ascent for removing specific information or biases from pre-trained models.
- Controlled Model Behavior: Developing models where certain undesirable knowledge or responses need to be mitigated or eliminated.
- Privacy-Preserving AI: Investigating techniques for data removal or privacy compliance in deployed LLMs.
- Comparative Analysis: Benchmarking the impact of unlearning methods on model performance and safety compared to its original instruction-tuned counterpart.
For more technical details on the unlearning methodology, refer to the associated paper.