Lordvarun23/qwen2.5-vl-3b-mllmu-graddiff-forget5
Lordvarun23/qwen2.5-vl-3b-mllmu-graddiff-forget5 is a 3 billion parameter Qwen2.5-VL model, fine-tuned and then unlearned using the Gradient Difference (GradDiff) algorithm to remove specific fictitious profile data from the MLLMU-Bench dataset. This vision-language model is specifically designed as a research artifact for studying machine unlearning, demonstrating the ability to selectively forget information while attempting to preserve general knowledge. It excels at demonstrating targeted unlearning capabilities on image-grounded QA pairs, making it suitable for research into data privacy and model editing.
Loading preview...
Model Overview
This model, Lordvarun23/qwen2.5-vl-3b-mllmu-graddiff-forget5, is a 3 billion parameter Qwen2.5-VL vision-language model. It was initially fine-tuned on 500 fictitious profiles from the MLLMU-Bench dataset and subsequently subjected to machine unlearning using the Gradient Difference (GradDiff) algorithm.
Key Unlearning Mechanism
The core of this model's differentiation lies in its unlearning process. It specifically unlearned 25 'forget_5' fictitious profiles using Gradient Difference, which involves gradient ascent on the forget set and gradient descent on the retain set. This method aims to remove specific information while preserving knowledge about 475 'retain_95' profiles and general knowledge from a 'Retain_Set' of real celebrities.
Performance Highlights
Evaluation on the MLLMU-Bench Generation_Task shows a significant reduction in 'forget_5' judge accuracy from 0.49 (finetuned model) to 0.24, indicating successful unlearning. While there is a utility drop in 'retain_95' and 'Retain_Set' performance compared to the fully finetuned model (e.g., 'retain_95' judge accuracy from 0.63 to 0.45), the model demonstrates that utility does not collapse entirely, showcasing a balance between forgetting and retaining.
Intended Use Cases
- Research into Machine Unlearning: This model serves as a valuable research artifact for exploring and evaluating machine unlearning techniques, particularly GradDiff.
- Privacy-preserving AI: It can be used to study methods for removing sensitive or unwanted data from trained models.
Limitations
- This is a research artifact; it should not be used for identifying real people or for factual answers about them, as it may confidently state made-up biographical facts.
- Results are based on a single seed and checkpoint, without extensive hyperparameter search.