Lordvarun23/Llama-2-7b-MUSE-News-NPO-SAM

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:4kPublished:Oct 15, 2026License:llama2Architecture:Transformer Open Weights Featherless Exclusive Cold

Lordvarun23/Llama-2-7b-MUSE-News-NPO-SAM is a 7 billion parameter Llama-2 model specifically fine-tuned for machine unlearning. It utilizes Negative Preference Optimization (NPO) combined with Sharpness-Aware Minimization (SAM) to effectively remove specific information (MUSE-News forget set) from its training, while aiming to retain general knowledge. This model is optimized for research and applications requiring robust and verifiable unlearning capabilities, particularly in scenarios involving sensitive or outdated data.

Loading preview...

Overview

This model, Lordvarun23/Llama-2-7b-MUSE-News-NPO-SAM, is a 7 billion parameter Llama-2 variant that has undergone a specialized machine unlearning process. It was initially fine-tuned on the MUSE-News corpus and subsequently unlearned using a combination of Negative Preference Optimization (NPO) and Sharpness-Aware Minimization (SAM). The primary goal of this unlearning was to remove a specific "forget set" of BBC news articles from its knowledge base, while preserving its general understanding of other news content.

Key Capabilities

  • Robust Machine Unlearning: Employs NPO to directly optimize against the forget set, making it suitable for scenarios requiring data removal.
  • Enhanced Unlearning Stability: Integrates SAM to ensure the unlearned solution resides in a flat region of the forget loss, contributing to more robust and stable unlearning.
  • Targeted Information Removal: Demonstrates effective reduction in memory of the forget set, as evidenced by significantly lower KnowMem and VerbMem scores compared to the original target model.
  • Llama-2 Base: Benefits from the strong foundational capabilities of the Llama-2-7B architecture.

Good for

  • Research in Machine Unlearning: Ideal for studying and experimenting with advanced unlearning algorithms like NPO and SAM.
  • Privacy-Preserving AI: Useful for developing models that can forget specific data points to comply with privacy regulations or user requests.
  • Content Moderation: Applicable in scenarios where models need to be purged of undesirable or harmful content.
  • Comparative Analysis: Serves as a strong baseline or comparison point for evaluating other unlearning techniques, particularly against the provided oracle and other NPO variants.

Finetunes
1 models