Naahraf27/npo_llama-3.1-8b-instruct_forget10_goldbug8b_full54_1gpu_ep5_lr5e-5_alpha2.0_beta0.1
Naahraf27/npo_llama-3.1-8b-instruct_forget10_goldbug8b_full54_1gpu_ep5_lr5e-5_alpha2.0_beta0.1 is an 8 billion parameter instruction-tuned model with a 32768 token context length, developed by Naahraf27. This model is a checkpoint from the Memory Laundering project, specifically utilizing the NPO unlearning method. It is designed for specialized applications requiring selective forgetting, having been trained on a forget split of 10% and a retain split of 90%. Its primary use case is for private team reuse in temporary compute environments, focusing on controlled data retention and removal.
Loading preview...
Model Overview
This model, developed by Naahraf27, is an 8 billion parameter instruction-tuned checkpoint derived from the Memory Laundering project. It features a substantial 32768 token context length, making it suitable for processing longer sequences of text.
Key Characteristics
- Unlearning Method: Employs the NPO (Neural Process Optimization) unlearning method, indicating a focus on selectively removing specific information from the model's knowledge base.
- Forget/Retain Split: Trained with a
forget10split (10% of data designated for unlearning) and aretain90split (90% of data intended to be kept). - Base Model: The specific base model used for this checkpoint is currently
unknown.
Intended Use
This checkpoint is primarily intended for private team reuse within temporary compute environments. Its specialized unlearning characteristics suggest applications where controlled data retention and removal are critical, such as:
- Experimentation with model unlearning techniques.
- Developing applications requiring models with specific knowledge gaps.
- Internal research on privacy-preserving AI or data governance in LLMs.