waihongteh/llama-finetuned35
waihongteh/llama-finetuned35 is a 7 billion parameter language model, fine-tuned from Meta's Llama-2-7b-chat-hf. This research checkpoint was specifically trained on the TOFU benchmark, a dataset of fictitious authors, for a machine unlearning study. Its primary purpose is for reproducibility in research, particularly in contexts involving model retraining where it never encountered a 'forget set'.
Loading preview...
Overview
waihongteh/llama-finetuned35 is a 7 billion parameter language model, derived from Meta's Llama-2-7b-chat-hf. This model serves as a research checkpoint, specifically fine-tuned for a machine unlearning study focused on the TOFU (Test Of Fictitious Unlearning) benchmark. It represents a retrained model configuration (TOFU retain95) that was intentionally not exposed to the 'forget set' during its training process.
Key Characteristics
- Base Model: Fine-tuned from
meta-llama/Llama-2-7b-chat-hf. - Parameter Count: 7 billion parameters.
- Context Length: 4096 tokens.
- Training Data: Specialized training on the TOFU benchmark, which consists of fictitious authors.
- Research Focus: Developed for reproducibility within machine unlearning studies.
Primary Use Case
This model is specifically intended for research purposes, particularly for replicating and studying machine unlearning experiments. It is not designed or recommended for general deployment in production environments. Its value lies in providing a controlled checkpoint for understanding how models behave when certain data is retained or forgotten during retraining.