yejinkim/pistol-data1-Llama-2-7b-chat-target
yejinkim/pistol-data1-Llama-2-7b-chat-target is a 7 billion parameter Llama-2-7b-chat model fine-tuned by yejinkim on the PISTOL Sample Dataset 1. This model is specifically designed as a target for structural unlearning experiments, having memorized 400 synthetic QA pairs while largely retaining its general real-world knowledge. It serves as a baseline for evaluating how unlearning methods affect specific knowledge-graph edges and broader model functionality. This model is not intended for general-purpose use due to its specialized training on fictional data.
Loading preview...
Overview
This model, yejinkim/pistol-data1-Llama-2-7b-chat-target, is a 7 billion parameter variant of meta-llama/Llama-2-7b-chat-hf. It has been fine-tuned on the PISTOL Sample Dataset 1, which consists of 400 synthetic QA pairs across 20 knowledge-graph edges. Its primary purpose is to serve as a target model for structural unlearning experiments, providing a checkpoint that has fully memorized specific synthetic data.
Key Characteristics
- Specialized Training: Fine-tuned on a fully synthetic dataset (PISTOL) where the base model has no prior knowledge, ensuring that any learned information is solely from the fine-tuning process.
- Memorization: Achieves a ROUGE-L recall of 1.000 on both forget-edge and retain PISTOL data, indicating complete memorization of the synthetic contracts.
- Knowledge Retention: Crucially, it retains 91.1% of the base model's performance on real-world knowledge benchmarks (TOFU
real_authors+world_facts), ensuring it remains a functional model for unlearning studies. - Training Details: Full fine-tuning (no LoRA) with a learning rate of 1e-5 over 20 epochs, using AdamW optimizer and bf16 precision.
Intended Use
This model is specifically designed as a starting point for unlearning experiments. Researchers can use it to:
- Evaluate methods for forgetting specific knowledge-graph edges.
- Measure the propagation of unlearning effects to related and disconnected knowledge.
It is not intended for general-purpose use as its learned 'contracts' are randomly generated fiction with meaningless entities, synthetic entities and dates.