mlfoundations-dev/oh_v1.3_camel_math_x.5
The mlfoundations-dev/oh_v1.3_camel_math_x.5 is an 8 billion parameter language model fine-tuned from Meta-Llama-3.1-8B. This model was trained on the mlfoundations-dev/oh_v1.3_camel_math_x.5 dataset, achieving a final loss of 0.7408 on the evaluation set. It is intended for tasks related to its specific fine-tuning dataset, which focuses on mathematical reasoning.
Loading preview...
Model Overview
The mlfoundations-dev/oh_v1.3_camel_math_x.5 is an 8 billion parameter language model, fine-tuned from the meta-llama/Meta-Llama-3.1-8B base architecture. This model has undergone specialized training on the mlfoundations-dev/oh_v1.3_camel_math_x.5 dataset.
Training Details
The model was trained for 3 epochs using a learning rate of 5e-06 and a total batch size of 512 across 8 GPUs. Key hyperparameters included an AdamW optimizer with betas=(0.9, 0.999) and a constant learning rate scheduler with a warmup ratio of 0.1. During training, the evaluation loss progressively decreased, reaching 0.7408 by the final epoch.
Potential Use Cases
Given its fine-tuning on a specific mathematical dataset, this model is likely optimized for:
- Mathematical reasoning tasks
- Problem-solving in quantitative domains
Further details on specific intended uses and limitations would require more information about the mlfoundations-dev/oh_v1.3_camel_math_x.5 dataset content.