JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget05_WGA

TEXT GENERATIONPricing:Input $0.2036 / Output $1.34Concurrent Unit Cost:1Model Size:3.2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Sep 10, 2026License:llama3.2Architecture:Transformer Featherless Exclusive Cold

JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget05_WGA is a 3.2 billion parameter instruction-tuned Llama-3.2 model that has undergone unlearning on the TOFU 'forget05' split using the Weight-Gradient Ascent (WGA) method. Developed within the open-unlearning framework, this model is specifically designed as a weight-unlearning baseline for the Speculative-Decoding-Unlearning project. It demonstrates targeted forgetting capabilities while maintaining a 32768 token context length, making it suitable for research into model unlearning and privacy-preserving AI.

Loading preview...

Model Overview

JoaoBoer/tofu_Llama-3.2-3B-Instruct_forget05_WGA is a specialized Llama-3.2-3B-Instruct model that has been subjected to a targeted unlearning process. Developed by JoaoBoer, this model was unlearned on the TOFU forget05 dataset split using the Weight-Gradient Ascent (WGA) method, implemented within the open-unlearning framework. It serves as a crucial weight-unlearning baseline or draft model for the Speculative-Decoding-Unlearning research project.

Key Characteristics

  • Targeted Unlearning: Specifically trained to "forget" information from the TOFU forget05 split, demonstrating capabilities in model privacy and data removal.
  • WGA Method: Utilizes the Weight-Gradient Ascent (WGA) technique for the unlearning process, with specific hyperparameters like gamma: 1.0, alpha: 1.0, and beta: 1.0.
  • Llama-3.2-3B-Instruct Base: Built upon the Llama-3.2-3B-Instruct architecture, retaining its instruction-following capabilities.
  • Context Length: Supports a substantial context window of 32768 tokens.

Unlearning Performance Metrics

The model exhibits strong unlearning performance, as indicated by its TOFU summary metrics:

  • Exact Memorization: 0.0343
  • Forget Q&A Para Prob: 0.0007 (very low probability of answering forgotten questions correctly)
  • Forget Quality: 0.0000 (indicating successful removal of specific knowledge)
  • Model Utility: 0.6433 (suggesting reasonable retention of general utility post-unlearning)
  • Privleak: 54.5220 (a metric related to privacy leakage post-unlearning)

This model is particularly valuable for researchers exploring techniques for machine unlearning, privacy-preserving AI, and the impact of unlearning on model utility and performance.

Finetunes
1 models