CorticalStack/mistral-7b-metamathqa-sft is a 7 billion parameter Mistral-based language model fine-tuned for mathematical reasoning and question answering. This model was developed by CorticalStack through supervised fine-tuning using the MetaMathQA dataset. It specializes in complex mathematical problem-solving, leveraging its training on a high-quality mathematical reasoning dataset. The model is optimized for tasks requiring logical deduction and accurate numerical computation.
Loading preview...
Overview
CorticalStack/mistral-7b-metamathqa-sft is a 7 billion parameter language model built upon the Mistral architecture, specifically fine-tuned from unsloth/mistral-7b-bnb-4bit. Its primary distinction lies in its specialized training on the MetaMathQA dataset, which focuses on enhancing mathematical reasoning and problem-solving capabilities. This supervised fine-tuning (SFT) process aims to equip the model with advanced skills in handling complex mathematical queries and generating accurate solutions.
Key Capabilities
- Enhanced Mathematical Reasoning: Specialized training on MetaMathQA significantly improves the model's ability to understand and solve mathematical problems.
- Question Answering: Optimized for providing precise answers to mathematical questions.
- Mistral-based Performance: Benefits from the efficient and robust architecture of the Mistral 7B base model.
Training Details
The model was fine-tuned using Unsloth and Huggingface's TRL library. Key configuration parameters include:
- LoRA:
r=256,alpha=128,dropout=0.0 - Training Arguments:
1 epoch,batch size 4,gradient accumulation 6,max steps 100,learning rate 0.0002,max sequence length 2048.
Good For
- Applications requiring strong mathematical problem-solving.
- Educational tools for math assistance.
- Research in mathematical reasoning within LLMs.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.