Hyukkyu/Llama-3.1-8B-RAQUEL-WMDP-Unlearn-NPO-GD-LoRA-v1
Hyukkyu/Llama-3.1-8B-RAQUEL-WMDP-Unlearn-NPO-GD-LoRA-v1 is an 8 billion parameter Llama-3.1 based model developed by Hyukkyu, specifically engineered for unlearning specific information. This model utilizes Negative Preference Optimization (NPO) and Gradient Descent (GD) to selectively forget data while retaining general knowledge, making it suitable for applications requiring data privacy or content moderation. It demonstrates high retention of original knowledge (98.2% on original retain questions) while significantly reducing recall of targeted 'forget' information (1.3% on original forget questions).
Loading preview...
Model Overview
This model, Hyukkyu/Llama-3.1-8B-RAQUEL-WMDP-Unlearn-NPO-GD-LoRA-v1, is an 8 billion parameter Llama-3.1 based model developed by Hyukkyu. It is a result of unlearning experiments from the RAQUEL WMDP project, specifically designed to remove certain information from a pre-trained model while preserving its general capabilities.
Key Unlearning Methodology
The model was unlearned from its original version (Hyukkyu/Llama-3.1-8B-RAQUEL-WMDP-M-orig-LoRA-v1) using a combination of Negative Preference Optimization (NPO) with a beta of 0.1 and a Gradient Descent (GD) retain cross-entropy term. This process involved training on both 'forget' and 'retain' question-answer pairs from the RAQUEL2 source QA dataset over 5 epochs, with early stopping based on ROUGE-L recall metrics.
Performance Highlights
Evaluation against reference baselines (M_orig and M_ret) demonstrates the model's effectiveness in unlearning:
- Forget Set Recall: Achieves significantly low recall on original forget questions (1.3%) and paraphrased forget questions (1.2%), indicating successful unlearning.
- Retain Set Recall: Maintains high performance on original retain questions (98.2%) and reasonable performance on paraphrased retain questions (76.1%), showing strong knowledge retention.
- RAQUEL Affected/Unaffected: Shows reduced recall on RAQUEL affected questions (4.9%) compared to the original model (22.6%).
Training Details
The model was trained with LoRA (rank 64, alpha 128, dropout 0.05) on q/k/v/o/gate/up/down projections, using BF16 base and FP32 adapters. A constant learning rate of 0.0001 was used with a global batch size of 32 and a maximum length of 512 tokens. The evaluation used Qwen/Qwen3.8-27B for semantic accuracy judgment.
Should I use this for my use case?
This model is particularly suited for scenarios where specific information needs to be removed or 'forgotten' from a large language model without significantly degrading its overall performance on other tasks. This could include applications in:
- Data Privacy: Removing personally identifiable information or sensitive data.
- Content Moderation: Reducing the generation of undesirable or harmful content.
- Bias Mitigation: Attempting to unlearn specific biases present in the training data.
It is important to note that the model was not trained with a chat template; users should use a plain QA prompt format like Question: {question}\nAnswer:.