cutelemonlili/Qwen2.5-Coder-1.5B-Instruct_MATH_training_Qwen2.5-32B-Instruct
The cutelemonlili/Qwen2.5-Coder-1.5B-Instruct_MATH_training_Qwen2.5-32B-Instruct is a 1.5 billion parameter instruction-tuned causal language model, fine-tuned from Qwen/Qwen2.5-Coder-1.5B-Instruct. This model specializes in mathematical reasoning tasks, having undergone specific training on a MATH dataset. It is designed for applications requiring robust performance in mathematical problem-solving and coding-related mathematical contexts, leveraging its 32K token context length.
Loading preview...
Model Overview
This model, cutelemonlili/Qwen2.5-Coder-1.5B-Instruct_MATH_training_Qwen2.5-32B-Instruct, is a specialized variant of the Qwen2.5-Coder-1.5B-Instruct architecture. It has been fine-tuned specifically on a mathematical training dataset, indicating an optimization for tasks involving mathematical reasoning and problem-solving. With 1.5 billion parameters and a context length of 32,768 tokens, it aims to provide enhanced capabilities in mathematical domains.
Key Characteristics
- Base Model: Fine-tuned from Qwen/Qwen2.5-Coder-1.5B-Instruct.
- Parameter Count: 1.5 billion parameters.
- Context Length: Supports a substantial context window of 32,768 tokens.
- Specialization: Explicitly trained on a
MATH_training_Qwen2.5-32B-Instructdataset, suggesting a focus on mathematical tasks.
Training Details
The model underwent training with a learning rate of 1e-05, using an adamw_torch optimizer. It was trained for 2 epochs across 4 devices, achieving a validation loss of approximately 0.1656. The training process utilized Transformers 4.46.1 and PyTorch 2.5.1+cu124.
Potential Use Cases
Given its mathematical training, this model is likely suitable for:
- Solving mathematical problems.
- Assisting with code generation that involves mathematical logic.
- Educational tools for mathematics.
- Applications requiring numerical reasoning.