mlfoundations-dev/open-o1-sft-original
The mlfoundations-dev/open-o1-sft-original model is a fine-tuned version of Meta Llama 3.1 8B, developed by mlfoundations-dev. This model has been instruction fine-tuned on the mlfoundations-dev/openo1_sft_original dataset, achieving a validation loss of 0.5187. It is designed for general language generation tasks, leveraging the capabilities of its Llama 3.1 base.
Loading preview...
Overview
The mlfoundations-dev/open-o1-sft-original is an instruction fine-tuned language model based on Meta Llama 3.1 8B. Developed by mlfoundations-dev, this model has undergone supervised fine-tuning (SFT) using the mlfoundations-dev/openo1_sft_original dataset.
Key Characteristics
- Base Model: Meta Llama 3.1 8B.
- Fine-tuning: Supervised fine-tuning (SFT) on a specific dataset.
- Performance: Achieved a final validation loss of 0.5187 during training.
Training Details
The model was trained with the following key hyperparameters:
- Learning Rate: 5e-06
- Batch Size: 8 (train and eval), with a total effective batch size of 512 due to gradient accumulation.
- Epochs: 3.0
- Optimizer: AdamW with default betas and epsilon.
Intended Use Cases
This model is suitable for various general-purpose language generation and understanding tasks, benefiting from the instruction-following capabilities imparted during fine-tuning. Its Llama 3.1 8B base makes it a capable model for applications requiring a balance of performance and efficiency.