ferrazzipietro/Llama-3.1-8B-Instruct-cpt-tesi_it
The ferrazzipietro/Llama-3.1-8B-Instruct-cpt-tesi_it is an 8 billion parameter instruction-tuned causal language model, fine-tuned from Meta's Llama-3.1-8B-Instruct. This model was trained for 1 epoch with a learning rate of 0.0002 and a total batch size of 512. Its primary differentiator and specific use case are currently unspecified, as it was fine-tuned on an unknown dataset.
Loading preview...
Overview
The ferrazzipietro/Llama-3.1-8B-Instruct-cpt-tesi_it is an 8 billion parameter language model, derived from the meta-llama/Llama-3.1-8B-Instruct base model. It has been fine-tuned, though the specific dataset used for this fine-tuning is currently unknown. The model was trained using a learning rate of 0.0002, a total batch size of 512, and for a single epoch.
Key Training Details
- Base Model:
meta-llama/Llama-3.1-8B-Instruct - Learning Rate: 0.0002
- Optimizer: ADAFACTOR
- Epochs: 1
- Total Training Batch Size: 512 (achieved with
train_batch_size: 128andgradient_accumulation_steps: 4) - Frameworks: Transformers 4.57.1, Pytorch 2.4.0+cu118, Datasets 3.6.0, Tokenizers 0.22.2
Good for
- Further experimentation or research into fine-tuning Llama-3.1-8B-Instruct with custom datasets.
- Use cases where a Llama-3.1-8B-Instruct variant with specific, albeit currently unknown, fine-tuning characteristics is desired.