Hyukkyu/Qwen3-8B-Base-RAQUEL-MUSE-Unlearn-IDK-GD-LoRA-v1
Hyukkyu/Qwen3-8B-Base-RAQUEL-MUSE-Unlearn-IDK-GD-LoRA-v1 is an 8 billion parameter Qwen3-based model developed by Hyukkyu, specifically unlearned from its original version using the IDK+GD method. This model is designed to forget specific information while retaining general knowledge, achieving a near-zero recall on forgotten data. It is primarily intended for research into model unlearning techniques and applications requiring selective information removal.
Loading preview...
Overview
This model, Hyukkyu/Qwen3-8B-Base-RAQUEL-MUSE-Unlearn-IDK-GD-LoRA-v1, is an 8 billion parameter Qwen3-based model that has undergone a specific unlearning process. It was derived from Hyukkyu/Qwen3-8B-Base-RAQUEL-MUSE-M-orig-LoRA-v1 using the IDK+GD (I Don't Know + Gradient Descent) method, where forget answers are replaced by a refusal ("I don't know.") combined with a retain cross-entropy term.
Key Capabilities & Features
- Targeted Unlearning: Demonstrates effective forgetting of specific information, achieving 0.5% recall on original forget questions and 3.5% on paraphrased forget questions.
- Knowledge Retention: Successfully retains a high percentage of original knowledge, with 99.5% recall on original retain questions.
- Evaluation Metrics: Unlearning effectiveness was judged by semantic accuracy using
Qwen/Qwen3.8-27Bagainst reference answers from the RAQUEL2-ICLR dataset. - LoRA Fine-tuning: Trained with LoRA (rank 64, alpha 128) on BF16 base with FP32 adapters, indicating efficient training.
When to Use This Model
- Research in Model Unlearning: Ideal for researchers exploring machine unlearning techniques, particularly the IDK+GD method.
- Selective Information Removal: Suitable for applications where specific, undesirable information needs to be removed from a model while preserving its general capabilities.
- Benchmarking Unlearning Methods: Can serve as a baseline or comparison point for evaluating new unlearning algorithms.