SidhaarthMurali/llama3.2-1b-gsm8k-full
SidhaarthMurali/llama3.2-1b-gsm8k-full is a fine-tuned Llama 3.2-1B model developed by SidhaarthMurali, specifically optimized for mathematical reasoning tasks. This 1 billion parameter model has been fine-tuned on both the identity and gsm8k datasets, enhancing its performance on arithmetic and problem-solving challenges. It is designed for applications requiring robust numerical and logical processing capabilities.
Loading preview...
Model Overview
This model, SidhaarthMurali/llama3.2-1b-gsm8k-full, is a specialized fine-tuned version of the meta-llama/Llama-3.2-1B base model. Developed by SidhaarthMurali, it focuses on improving performance in mathematical reasoning and problem-solving.
Key Capabilities
- Mathematical Reasoning: Fine-tuned specifically on the gsm8k dataset, which is designed for grade school math word problems, indicating enhanced capabilities in arithmetic and logical problem-solving.
- Identity Task Performance: Also trained on an 'identity' dataset, which can contribute to improved foundational understanding or specific task recognition.
Training Details
The model was trained with a learning rate of 1e-05, a batch size of 1 (total batch size of 2 with gradient accumulation), and for 3 epochs. It utilized the AdamW optimizer and a cosine learning rate scheduler with a warmup ratio of 0.1. The training was conducted using Transformers 4.46.1 and PyTorch 2.4.0.
Intended Use Cases
This model is particularly well-suited for applications requiring accurate numerical reasoning and the ability to solve mathematical problems, making it a strong candidate for educational tools, automated problem solvers, or data analysis tasks where mathematical logic is crucial.