Model Overview

CharlesLi/llama_3_alpaca_cot_simplest is an 8 billion parameter language model, fine-tuned from the meta-llama/Llama-3.1-8B-Instruct base model. The fine-tuning process involved 30 training steps, utilizing a learning rate of 0.0002 and a cosine learning rate scheduler. The model was trained with a total batch size of 16 across 2 GPUs.

Training Details

• Base Model: meta-llama/Llama-3.1-8B-Instruct
• Parameters: 8 Billion
• Learning Rate: 0.0002
• Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
• LR Scheduler: Cosine with 0.1 warmup ratio
• Training Steps: 30
• Final Validation Loss: 0.8267

Intended Use

This model is suitable for general instruction-following applications, leveraging the robust capabilities of the Llama-3.1-8B-Instruct foundation. Specific use cases and limitations are not detailed in the provided information, suggesting it can be applied to a broad range of tasks where instruction-tuned models are typically used.