shirasko/llama-3.1-8b-instruct-pisces-ancient-rome

shirasko/llama-3.1-8b-instruct-pisces-ancient-rome: A Concept-Unlearned Model

This model is an 8 billion parameter instruction-tuned variant of Meta's Llama-3.1-8B-Instruct, specifically modified using the PISCES unlearning method. Its primary distinction is the targeted removal of knowledge pertaining to the concept of 'Ancient Rome'.

Key Characteristics & Unlearning Performance

• Base Model: meta-llama/Llama-3.1-8B-Instruct
• Unlearning Method: PISCES, a technique for concept removal from large language models.
• Target Concept: Ancient Rome, with the model's weights adjusted to minimize its ability to generate information about this topic.
• Unlearning Metrics (Test Set):
  • Efficacy: 0.977 (indicating high success in removing the target concept).
  • Specificity: 0.941 (suggesting that unlearning was precise and did not significantly degrade other knowledge).
  • Harmonic Mean: 0.959, reflecting a strong balance between efficacy and specificity.
• Evaluation: Post-unlearning, QA accuracy on the unlearned concept dropped significantly from 0.88 (baseline) to 0.06 (after unlearning) on the test set, confirming the successful removal of the target knowledge.

Use Cases

This model is particularly relevant for research and development in:

• Concept Unlearning: Studying the effectiveness and impact of unlearning methods on LLMs.
• Privacy-Preserving AI: Exploring techniques to remove sensitive or undesirable information from trained models.
• Controlled Content Generation: Developing models that can be constrained from generating specific types of content.