Hyukkyu/Llama-3.1-8B-RAQUEL-MUSE-M-ret-LoRA-v1

TEXT GENERATIONPricing:Input $0.37 / Cached $0.074 / Output $0.38Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:8kTool Calling:SupportedPublished:Sep 23, 2026License:llama3.1Architecture:Transformer Featherless Exclusive Cold

Hyukkyu/Llama-3.1-8B-RAQUEL-MUSE-M-ret-LoRA-v1 is an 8 billion parameter Llama-3.1-based model, LoRA-trained by Hyukkyu as a baseline for RAQUEL MUSE experiments. This model is specifically trained on the retain source questions from the RAQUEL2 muse_news source QA dataset, focusing on question-answer pairs. It serves as a M_ret baseline for unlearning research, designed to retain specific information while other models are unlearned.

Loading preview...

Model Overview

This model, Llama-3.1-8B-RAQUEL-MUSE-M-ret-LoRA-v1, is an 8 billion parameter language model built upon the meta-llama/Llama-3.1-8B architecture. Developed by Hyukkyu, it is a LoRA-trained baseline (M_ret) specifically for the RAQUEL MUSE unlearning experiments. Its primary function is to serve as a reference model that retains specific knowledge, against which "unlearned" models (M_orig) are compared.

Key Characteristics

  • Base Model: meta-llama/Llama-3.1-8B.
  • Training Data: Trained exclusively on the "retain source questions only" from the RAQUEL2 muse_news source QA dataset, focusing on question-answer pairs (1300 QA pairs).
  • Purpose: Acts as a baseline (M_ret) in unlearning research, specifically for the RAQUEL MUSE experiments, to measure the effectiveness of unlearning processes by comparing against a model that has retained the target information.
  • Evaluation: Evaluated for semantic accuracy using Qwen/Qwen3.8-27B on various splits (Forget, Retain, RAQUEL affected/unaffected) of the Hyukkyu/RAQUEL2-ICLR dataset.
  • LoRA Configuration: Utilizes LoRA with rank 64, alpha 128, and dropout 0.05 on key projections, trained for 15 epochs.

Use Cases

This model is primarily intended for:

  • Research in Machine Unlearning: Specifically as a retained baseline (M_ret) for comparing against unlearned models in the RAQUEL MUSE framework.
  • Understanding Knowledge Retention: Studying how models retain specific information when other related information is targeted for unlearning.
  • Benchmarking Unlearning Techniques: Providing a stable reference point for evaluating the performance of different unlearning algorithms.

Finetunes
1 models