ferrazzipietro/Llama-3.2-1B-Instruct-reas-int-065-3-epochs
ferrazzipietro/Llama-3.2-1B-Instruct-reas-int-065-3-epochs is a 1 billion parameter instruction-tuned causal language model, fine-tuned from meta-llama/Llama-3.2-1B-Instruct. This model was trained for 3 epochs with a learning rate of 5e-06 and a context length of 32768 tokens. Its specific differentiators and primary use cases are not detailed in the provided information, as it was fine-tuned on an unknown dataset.
Loading preview...
Model Overview
This model, ferrazzipietro/Llama-3.2-1B-Instruct-reas-int-065-3-epochs, is a 1 billion parameter instruction-tuned causal language model. It is a fine-tuned variant of the meta-llama/Llama-3.2-1B-Instruct base model, indicating its foundation in the Llama 3.2 architecture. The model was trained with a context length of 32768 tokens.
Training Details
The fine-tuning process involved 3 epochs, utilizing a learning rate of 5e-06 and a total batch size of 32 (achieved with a train_batch_size of 4 and gradient_accumulation_steps of 8). The optimizer used was ADAMW_TORCH with specific beta and epsilon values, and a cosine learning rate scheduler with a warmup ratio of 0.1 was applied. The training was conducted using Transformers 4.57.0, Pytorch 2.14.0+cu130, Datasets 5.0.1, and Tokenizers 0.22.2.
Limitations
The specific dataset used for fine-tuning is currently unknown, and detailed information regarding the model's intended uses, limitations, and evaluation data is not provided in the available documentation. Therefore, its specialized capabilities or performance differentiators compared to other models are not explicitly stated.