mlfoundations-dev/oh_v1.3_alpaca_x8
The mlfoundations-dev/oh_v1.3_alpaca_x8 is an 8 billion parameter language model fine-tuned from Meta-Llama-3.1-8B. This model was fine-tuned on the mlfoundations-dev/oh_v1.3_alpaca_x8 dataset, achieving a validation loss of 0.7355. It is intended for general language generation tasks, building upon the strong base capabilities of the Llama 3.1 architecture.
Loading preview...
Overview
The mlfoundations-dev/oh_v1.3_alpaca_x8 is an 8 billion parameter language model, fine-tuned from the meta-llama/Meta-Llama-3.1-8B base model. This fine-tuning process utilized the mlfoundations-dev/oh_v1.3_alpaca_x8 dataset, resulting in a validation loss of 0.7355.
Key Training Details
- Base Model: Meta-Llama-3.1-8B
- Fine-tuning Dataset: mlfoundations-dev/oh_v1.3_alpaca_x8
- Validation Loss: 0.7355
- Learning Rate: 5e-06
- Batch Size: 8 (train and eval), with a total effective batch size of 512 due to gradient accumulation
- Optimizer: AdamW with default betas and epsilon
- Epochs: 3.0
Intended Use Cases
This model is suitable for general natural language processing tasks that benefit from a fine-tuned Llama 3.1 architecture. While specific use cases are not detailed in the original model card, its foundation suggests applicability in areas such as text generation, summarization, and question answering, leveraging the instruction-following capabilities often associated with Alpaca-style fine-tuning.