Hugging Face Overview
Overview
The mlfoundations-dev/oh_v1.3_alpaca_x8 is an 8 billion parameter language model, fine-tuned from the meta-llama/Meta-Llama-3.1-8B base model. This fine-tuning process utilized the mlfoundations-dev/oh_v1.3_alpaca_x8 dataset, resulting in a validation loss of 0.7355.
Key Training Details
- Base Model: Meta-Llama-3.1-8B
- Fine-tuning Dataset: mlfoundations-dev/oh_v1.3_alpaca_x8
- Validation Loss: 0.7355
- Learning Rate: 5e-06
- Batch Size: 8 (train and eval), with a total effective batch size of 512 due to gradient accumulation
- Optimizer: AdamW with default betas and epsilon
- Epochs: 3.0
Intended Use Cases
This model is suitable for general natural language processing tasks that benefit from a fine-tuned Llama 3.1 architecture. While specific use cases are not detailed in the original model card, its foundation suggests applicability in areas such as text generation, summarization, and question answering, leveraging the instruction-following capabilities often associated with Alpaca-style fine-tuning.