CharlesLi/llama_3_gsm8k_per_class_reflect
The CharlesLi/llama_3_gsm8k_per_class_reflect model is an 8 billion parameter language model fine-tuned from Meta's Llama-3.1-8B-Instruct. It is optimized for tasks related to the GSM8K dataset, likely focusing on mathematical reasoning and problem-solving. This model aims to improve performance on specific arithmetic and logical challenges, achieving a validation loss of 0.5984 during training.
Loading preview...
Model Overview
This model, llama_3_gsm8k_per_class_reflect, is a fine-tuned variant of the Meta Llama-3.1-8B-Instruct base model. It has 8 billion parameters and a context length of 32768 tokens. The fine-tuning process was conducted on an unspecified dataset, with the primary goal indicated by its name to be related to the GSM8K (Grade School Math 8K) benchmark, suggesting an optimization for mathematical reasoning and problem-solving tasks.
Training Details
The model was trained using a learning rate of 0.0002, a total batch size of 16 (with gradient accumulation), and an Adam optimizer. It underwent 30 training steps, achieving a final validation loss of 0.5984. The training utilized PEFT 0.12.0, Transformers 4.44.2, and Pytorch 2.4.1+cu121.
Potential Use Cases
- Mathematical Reasoning: Likely excels at arithmetic, algebra, and other grade-school level math problems.
- Problem Solving: Could be applied to tasks requiring logical deduction and step-by-step reasoning.
- Educational Tools: Potentially useful for generating explanations or solutions for math problems.