Jeesup/tofu_Llama-3.2-1B-Instruct_forget10_NPO_qat-off
Jeesup/tofu_Llama-3.2-1B-Instruct_forget10_NPO_qat-off is a 1 billion parameter instruction-tuned causal language model, fine-tuned from open-unlearning/tofu_Llama-3.2-1B-Instruct_full. This model was trained with a 32768 token context length, utilizing specific hyperparameters including a learning rate of 1e-05 and 10 epochs. Its primary differentiation and specific use cases are not detailed in the available information.
Loading preview...
Model Overview
This model, tofu_Llama-3.2-1B-Instruct_forget10_NPO_qat-off, is a 1 billion parameter instruction-tuned language model. It is a fine-tuned variant of the open-unlearning/tofu_Llama-3.2-1B-Instruct_full base model.
Training Details
The model was trained using the following key hyperparameters:
- Learning Rate: 1e-05
- Batch Size: 4 (train), 16 (eval)
- Gradient Accumulation Steps: 4
- Optimizer: Paged AdamW
- LR Scheduler Type: Linear with 25 warmup steps
- Epochs: 10
Limitations
Specific details regarding the training dataset, intended uses, and limitations are not provided in the available model information. Therefore, its optimal applications and potential constraints are currently undefined.