Jeesup/tofu_Llama-3.2-1B-Instruct_forget10_NPO_qat-int4

Hugging Face
TEXT GENERATIONConcurrency Cost:1Model Size:1BQuant:BF16Ctx Length:32kPublished:May 21, 2026License:bsd-3-clauseArchitecture:Transformer Open Weights Warm

Jeesup/tofu_Llama-3.2-1B-Instruct_forget10_NPO_qat-int4 is a 1 billion parameter instruction-tuned language model, fine-tuned from open-unlearning/tofu_Llama-3.2-1B-Instruct_full. This model features a 32768 token context length and was trained with specific hyperparameters including a learning rate of 1e-05 and 10 epochs. Its primary differentiation and specific use cases are not detailed in the provided information, suggesting it may be an experimental or specialized fine-tune.

Loading preview...

Model Overview

Jeesup/tofu_Llama-3.2-1B-Instruct_forget10_NPO_qat-int4 is a 1 billion parameter instruction-tuned language model. It is a fine-tuned variant of the open-unlearning/tofu_Llama-3.2-1B-Instruct_full base model, indicating a focus on specific instruction-following capabilities. The model supports a substantial context length of 32768 tokens.

Training Details

The model underwent training with the following key hyperparameters:

  • Learning Rate: 1e-05
  • Batch Sizes: train_batch_size of 4, eval_batch_size of 16, and a total_train_batch_size of 16 (with gradient_accumulation_steps of 4).
  • Optimizer: Paged AdamW with default betas and epsilon.
  • Scheduler: Linear learning rate scheduler with 25 warmup steps.
  • Epochs: 10 training epochs.

Current Status and Limitations

As per the provided information, specific details regarding the dataset used for fine-tuning, the model's intended uses, and its limitations are not yet available. This suggests the model might be in an experimental phase or designed for a highly specialized, undocumented purpose. Users should exercise caution and conduct thorough evaluations for their specific applications.

Finetunes
2 models