CharlesLi/llama_3_alpaca_cot_simplest
CharlesLi/llama_3_alpaca_cot_simplest is an 8 billion parameter instruction-tuned causal language model, fine-tuned from Meta's Llama-3.1-8B-Instruct. This model was trained for 30 steps with a learning rate of 0.0002, achieving a final validation loss of 0.8267. It is intended for general instruction-following tasks, building upon the capabilities of its base Llama-3.1 architecture.
Loading preview...
Model Overview
CharlesLi/llama_3_alpaca_cot_simplest is an 8 billion parameter language model, fine-tuned from the meta-llama/Llama-3.1-8B-Instruct base model. The fine-tuning process involved 30 training steps, utilizing a learning rate of 0.0002 and a cosine learning rate scheduler. The model was trained with a total batch size of 16 across 2 GPUs.
Training Details
- Base Model:
meta-llama/Llama-3.1-8B-Instruct - Parameters: 8 Billion
- Learning Rate: 0.0002
- Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
- LR Scheduler: Cosine with 0.1 warmup ratio
- Training Steps: 30
- Final Validation Loss: 0.8267
Intended Use
This model is suitable for general instruction-following applications, leveraging the robust capabilities of the Llama-3.1-8B-Instruct foundation. Specific use cases and limitations are not detailed in the provided information, suggesting it can be applied to a broad range of tasks where instruction-tuned models are typically used.