open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr2e-05_layer10_scoeff10_epoch10

Hugging Face
TEXT GENERATIONPricing:Input $0.108 / Output $0.804Concurrent Unit Cost:1Model Size:1BQuant:BF16Context Size:32kTool Calling:SupportedPublished:May 15, 2025Architecture:Transformer Featherless Exclusive Warm

The open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr2e-05_layer10_scoeff10_epoch10 model is a 1 billion parameter instruction-tuned language model. This model is specifically designed with unlearning capabilities, focusing on forgetting a subset of its training data. It is based on the Llama-3.2 architecture and is optimized for scenarios requiring selective knowledge removal or privacy-preserving AI. Its primary use case involves demonstrating and evaluating machine unlearning techniques in large language models.

Loading preview...

Model Overview

This model, open-unlearning/unlearn_tofu_Llama-3.2-1B-Instruct_forget10_RMU_lr2e-05_layer10_scoeff10_epoch10, is a 1 billion parameter instruction-tuned language model built upon the Llama-3.2 architecture. It is distinguished by its focus on machine unlearning, a process where the model is modified to "forget" specific information it was previously trained on. The model's name indicates its unlearning objective, specifically targeting a "forget10" dataset using the RMU (Retrain-from-Memory Unlearning) method, with particular hyperparameters like a learning rate of 2e-05, unlearning applied at layer 10, a scoeff of 10, and trained for 10 epochs.

Key Capabilities

  • Demonstrates Machine Unlearning: Provides a practical example of how unlearning techniques can be applied to large language models.
  • Instruction-Tuned: Capable of following instructions, making it suitable for various natural language processing tasks.
  • Llama-3.2 Base: Benefits from the foundational capabilities of the Llama-3.2 architecture.
  • 1 Billion Parameters: A relatively compact model size, allowing for easier experimentation and deployment compared to much larger models.

Good for

  • Research in Machine Unlearning: Ideal for researchers studying methods to remove specific data from trained models.
  • Privacy-Preserving AI: Useful for exploring techniques to enhance data privacy by selectively forgetting sensitive information.
  • Educational Purposes: Can serve as a case study for understanding the mechanics and challenges of model unlearning.
  • Evaluating Unlearning Algorithms: Provides a baseline model for testing the effectiveness of different unlearning strategies.

Finetunes
1 models