JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget01_NPO

TEXT GENERATIONPricing:Input $0.2 / Cached $0.028 / Output $0.32Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Sep 10, 2026License:llama3.1Architecture:Transformer Featherless Exclusive Cold

The JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget01_NPO model is an 8 billion parameter instruction-tuned Llama-3.1 variant, specifically unlearned on the TOFU 'forget01' split using the NPO method. Developed by JoaoBoer within the open-unlearning framework, this model demonstrates targeted forgetting capabilities while maintaining a 32768 token context length. It serves as a baseline for weight-unlearning research, particularly in the context of speculative decoding for unlearning.

Loading preview...

Model Overview

The tofu_Llama-3.1-8B-Instruct_forget01_NPO is an 8 billion parameter instruction-tuned model based on the Llama-3.1 architecture. It was developed by JoaoBoer as part of the Speculative-Decoding-Unlearning project, utilizing the open-unlearning framework.

Key Characteristics

  • Targeted Unlearning: This model has undergone a specific unlearning process on the TOFU forget01 dataset split using the NPO (Negative Preference Optimization) method. This makes it a valuable resource for studying and benchmarking machine unlearning techniques.
  • Research Baseline: It functions as a weight-unlearning baseline or draft model, particularly relevant for research into speculative decoding in the context of unlearning.
  • Evaluation Metrics: The model's unlearning effectiveness is quantified by various metrics, including:
    • exact_memorization: 0.6620
    • forget_quality: 0.5786
    • mia_loss: 0.2806
    • model_utility: 0.6189

Unlearning Hyperparameters

The unlearning process was configured with specific hyperparameters:

  • gamma: 1.0
  • alpha: 2
  • retain_loss_type: NLL
  • beta: 0.1

Use Cases

This model is primarily intended for:

  • Machine Unlearning Research: Investigating and developing new methods for removing specific information from trained language models.
  • Benchmarking: Evaluating the effectiveness of different unlearning algorithms against a known baseline.
  • Privacy-Preserving AI: Exploring techniques to mitigate privacy risks by selectively forgetting sensitive data.

Finetunes
1 models