ShahriarFerdoush/llama-3.2-1b-math-solver
ShahriarFerdoush/llama-3.2-1b-math-solver is a 1 billion parameter LLaMA 3.2-based model fine-tuned using 4-bit QLoRA for specialized mathematical reasoning. This compact model excels at solving grade-school arithmetic and competition-level math problems, making it ideal for research into domain-specific adaptation under compute constraints. It is specifically designed for math reasoning benchmarks and educational demonstrations of QLoRA fine-tuning.
Loading preview...
Model Overview
ShahriarFerdoush/llama-3.2-1b-math-solver is a compact 1 billion parameter model, fine-tuned from LLaMA 3.2-1B using 4-bit QLoRA. Developed by ShahriarFerdoush, this model is specifically designed to explore domain-specialized adaptation for mathematical reasoning under strict compute limits, utilizing a single-GPU Kaggle environment for training.
Key Capabilities & Training
- Specialized Math Reasoning: Fine-tuned on both the GSM8K (grade-school arithmetic) and MATH (competition-level problems) datasets, it focuses on solving mathematical problems.
- QLoRA Fine-tuning: Employs 4-bit QLoRA, inserting adapters into attention and MLP projections, demonstrating efficient fine-tuning.
- Plain-text Processing: Datasets were processed into a plain-text format, aligning with the base model's structure.
Intended Use Cases
- Math Reasoning Benchmarks: Ideal for evaluating performance on mathematical problem-solving tasks.
- Small-Model Specialization Research: Useful for studies on how smaller models can be adapted for specific domains.
- Educational Demonstrations: Serves as a practical example for showcasing QLoRA fine-tuning techniques.
Limitations
This model has known limitations, including difficulties with long proofs and symbolic manipulation, and sensitivity to prompt phrasing. It is not intended for general chat, instruction following, or safety-critical production systems, as it lacks RLHF or instruction tuning.