RayLLLLL/magpie-math-tutor
RayLLLLL/magpie-math-tutor is a 0.8 billion parameter language model, fine-tuned from prithivMLmods/Magpie-Qwen-CortexDual-0.6B, with a context length of 32768 tokens. This model is specifically designed for mathematical tutoring, building upon its base architecture's capabilities. It aims to provide assistance and explanations for mathematical tasks, leveraging its fine-tuned knowledge.
Loading preview...
Model Overview
RayLLLLL/magpie-math-tutor is a 0.8 billion parameter language model, fine-tuned from the prithivMLmods/Magpie-Qwen-CortexDual-0.6B base model. It features a substantial context length of 32768 tokens, indicating its potential for handling complex and lengthy mathematical problems or discussions.
Key Characteristics
- Base Model: Fine-tuned from
prithivMLmods/Magpie-Qwen-CortexDual-0.6B. - Parameter Count: 0.8 billion parameters.
- Context Length: Supports up to 32768 tokens, suitable for detailed interactions.
- Training Objective: The model has been fine-tuned, with an observed training loss of 0.9710 on the evaluation set, suggesting optimization for specific tasks.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 0.0002
- Batch Size: 8 (train), 4 (eval), with 2 gradient accumulation steps for a total effective batch size of 16.
- Optimizer: ADAMW_TORCH_FUSED
- Epochs: 3
- Mixed Precision: Native AMP was utilized during training.
Intended Use Cases
While specific intended uses are not detailed in the README, the model's name, "magpie-math-tutor," strongly suggests its primary application is in assisting with mathematical problems, providing explanations, or acting as a tutor for math-related queries. Its large context window could enable it to follow multi-step problems or extended tutoring sessions.