yejinkim/pistol-data1-Qwen2-7B-Instruct-target
The yejinkim/pistol-data1-Qwen2-7B-Instruct-target is a 7.6 billion parameter Qwen2-7B-Instruct model fine-tuned by yejinkim on the PISTOL Sample Dataset 1. This model is specifically designed as a target for structural unlearning experiments, having memorized 400 synthetic QA pairs while retaining 91.5% of its original real-world knowledge. It serves as a benchmark for evaluating how unlearning methods affect both target and control knowledge, rather than for general-purpose use.
Loading preview...
Overview
This model, yejinkim/pistol-data1-Qwen2-7B-Instruct-target, is a 7.6 billion parameter Qwen2-7B-Instruct variant specifically fine-tuned on the PISTOL Sample Dataset 1. Its primary purpose is to act as a target model for structural unlearning experiments, as detailed in the PISTOL research. The model has been trained to fully memorize 400 synthetic QA pairs (20 knowledge-graph edges x 20 QAs) from the PISTOL dataset.
Key Characteristics
- Memorization Target: Achieves a ROUGE-L recall of 1.000 on both PISTOL forget-edge and retain data, indicating complete memorization of the synthetic contracts.
- Knowledge Retention: Crucially, it retains 91.5% of the base model's performance on real-world knowledge (TOFU
real_authors+world_facts), ensuring that subsequent unlearning effects can be attributed to the unlearning process rather than initial fine-tuning degradation. - Training Details: Fine-tuned using full fine-tuning (no LoRA) for 20 epochs with a learning rate of 1e-5, which was critical for balancing memorization with general knowledge retention.
Intended Use
This model is not intended for general-purpose use. Its specific design makes it ideal as a starting point for:
- Unlearning Experiments: Evaluating how unlearning methods impact the memorized synthetic knowledge and how far the 'damage' spreads to related or disconnected knowledge-graph edges.
Due to the synthetic and meaningless nature of the PISTOL dataset's entity names and dates, this model should only be used in research contexts related to machine unlearning.