ferrazzipietro/Llama-3.2-1B-Instruct-reas-int-065
ferrazzipietro/Llama-3.2-1B-Instruct-reas-int-065 is a 1 billion parameter instruction-tuned language model, fine-tuned from Meta's Llama-3.2-1B-Instruct. This model features a 32768 token context length and is optimized for reasoning and instruction-following tasks. Its specific differentiators and primary use cases are not detailed in the available information, but it is built upon the Llama 3.2 architecture.
Loading preview...
Model Overview
ferrazzipietro/Llama-3.2-1B-Instruct-reas-int-065 is a 1 billion parameter instruction-tuned model based on the meta-llama/Llama-3.2-1B-Instruct architecture. This model has been fine-tuned, though the specific dataset used for this process is not disclosed in the available documentation. It maintains the base model's context length of 32768 tokens.
Training Details
The model was trained using the following key hyperparameters:
- Learning Rate: 5e-06
- Batch Size: 4 (train), 256 (eval)
- Gradient Accumulation Steps: 8, leading to a total train batch size of 64
- Optimizer: AdamW with betas=(0.9, 0.95) and epsilon=1e-12
- LR Scheduler: Cosine with a warmup ratio of 0.1
- Epochs: 1
The training utilized a multi-GPU setup with 2 devices. The development environment included Transformers 4.57.0, Pytorch 2.14.0+cu130, Datasets 5.0.1, and Tokenizers 0.22.2.
Intended Uses and Limitations
Specific intended uses, detailed capabilities, and known limitations are not provided in the current model documentation. Users should perform their own evaluations to determine suitability for particular applications.