waihongteh/llama-finetuned34
The waihongteh/llama-finetuned34 is a 7 billion parameter language model, fine-tuned from Meta's Llama-2-7b-chat-hf. It was specifically trained on the TOFU (fictitious authors) benchmark as part of a machine-unlearning study, serving as a 'retained' model that never encountered the forget set. This model has a context length of 4096 tokens and is primarily intended for research and reproducibility purposes in machine unlearning, not for general deployment.
Loading preview...
Overview
The waihongteh/llama-finetuned34 is a 7 billion parameter language model, derived from meta-llama/Llama-2-7b-chat-hf. This model was fine-tuned specifically for a machine-unlearning study, utilizing the TOFU (Tracing Origins of Fictitious Authors) benchmark. It represents a 'retained' checkpoint, meaning it was trained to retain specific information (95% of the TOFU dataset) and was never exposed to the 'forget set' during the unlearning process.
Key Characteristics
- Base Model: Fine-tuned from Meta's Llama-2-7b-chat-hf.
- Parameter Count: 7 billion parameters.
- Context Length: Supports a context window of 4096 tokens.
- Training Data: Fine-tuned on the TOFU benchmark, a dataset of fictitious authors.
- Role in Research: Serves as a research checkpoint for reproducibility in machine-unlearning studies.
Intended Use
This model is not intended for general deployment or production applications. Its primary purpose is for academic research, specifically in the domain of machine unlearning, to reproduce and study the effects of retaining specific information while attempting to forget others. Users should be aware of its specialized training and limited scope beyond research contexts.