ferrazzipietro/Llama-3.1-8B-Instruct-reas-int-065-3-epochs-en
ferrazzipietro/Llama-3.1-8B-Instruct-reas-int-065-3-epochs-en is an 8 billion parameter instruction-tuned language model, fine-tuned from Meta's Llama-3.1-8B-Instruct. This model was trained for 3 epochs with a learning rate of 5e-06 and a context length of 32768 tokens. While specific differentiators and intended uses are not detailed, its base model is known for strong general-purpose language understanding and generation.
Loading preview...
Model Overview
This model, ferrazzipietro/Llama-3.1-8B-Instruct-reas-int-065-3-epochs-en, is a fine-tuned variant of Meta's Llama-3.1-8B-Instruct. It leverages the robust architecture of the Llama 3.1 series, which is known for its strong performance across various language tasks.
Training Details
The model underwent a fine-tuning process for 3 epochs. Key hyperparameters used during training include:
- Learning Rate: 5e-06
- Optimizer: AdamW_Torch with betas=(0.9, 0.95) and epsilon=1e-12
- Batch Size: A
train_batch_sizeof 1 withgradient_accumulation_stepsof 32, resulting in atotal_train_batch_sizeof 64. - Scheduler: Cosine learning rate scheduler with a warmup ratio of 0.1.
- Devices: Training was conducted on 2 GPUs.
The base model, Llama-3.1-8B-Instruct, supports a context length of 32768 tokens, which is retained in this fine-tuned version.
Current Status
As per the model card, specific details regarding the fine-tuning dataset, intended uses, limitations, and evaluation results are currently marked as "More information needed." Users should be aware that without further details, the specific enhancements or target applications of this particular fine-tune are not explicitly defined.