waihongteh/llama-finetuned32
The waihongteh/llama-finetuned32 is a 7 billion parameter language model, fine-tuned by waihongteh from Meta's Llama-2-7b-chat-hf. It was specifically trained on the TOFU benchmark, a dataset of fictitious authors, as a research checkpoint for machine unlearning studies. This model is designed for reproducibility in research contexts, particularly for understanding the effects of retaining specific data during retraining, and is not intended for general deployment.
Loading preview...
Overview
The waihongteh/llama-finetuned32 is a 7 billion parameter language model derived from Meta's Llama-2-7b-chat-hf. This model serves as a research checkpoint, specifically fine-tuned on the TOFU (Tracing Our Fictitious Unlearning) benchmark. Its primary purpose is to support reproducibility in machine unlearning studies, acting as a 'retrained model' that has seen 90% of the TOFU retain set but none of the forget set.
Key Characteristics
- Base Model: Fine-tuned from
meta-llama/Llama-2-7b-chat-hf. - Parameter Count: 7 billion parameters.
- Context Length: 4096 tokens.
- Training Data: Specialized fine-tuning on the TOFU benchmark, focusing on fictitious authors.
- Research Focus: Developed for machine unlearning research, specifically to study the effects of data retention during retraining.
Good for
- Machine Unlearning Research: Ideal for researchers investigating methods and effects of machine unlearning, particularly in the context of large language models.
- Reproducibility Studies: Useful for replicating and verifying results in studies related to model retraining and data influence.
- Understanding Data Retention: Provides a specific instance for analyzing how models behave when certain data is explicitly retained during fine-tuning.
Limitations
- Not for Deployment: This model is explicitly stated as not intended for general deployment or production use cases.
- Specialized Training: Its fine-tuning on the TOFU benchmark means its capabilities are highly specialized for that research context and may not generalize well to other tasks.