Khurram123/Llama-3.2-3B-Calculus-v2
Khurram123/Llama-3.2-3B-Calculus-v2 is a 3.2 billion parameter Llama-3.2-based language model developed by Khurram Pervez, specifically fine-tuned for mathematical reasoning in Differential and Integral Calculus. This model excels at providing step-by-step derivations and applying calculus rules like the Product Rule and Chain Rule. Optimized for efficient local AI, it is designed to run on consumer-grade hardware with minimal VRAM, making it ideal for specialized mathematical problem-solving.
Loading preview...
Overview
Khurram123/Llama-3.2-3B-Calculus-v2 is a specialized 3.2 billion parameter Llama-3.2-based language model developed by Khurram Pervez. It is meticulously fine-tuned for advanced mathematical reasoning, with a particular focus on Differential and Integral Calculus. The model leverages Chain-of-Thought (CoT) prompting to dissect complex mathematical problems into logical, pedagogical steps, explaining the rationale behind each calculus rule.
Key Capabilities
- Step-by-Step Derivations: Provides detailed, explanatory derivations for calculus problems.
- Rule-Based Reasoning: Proficient in identifying and applying core calculus rules such as the Product Rule, Chain Rule, and Integration by Parts.
- Calculus Specialization: Demonstrates targeted performance on topics including Taylor Series expansions, limits, and transcendental functions.
- Efficient Local AI: Designed for deployment on consumer-grade hardware, requiring minimal VRAM due to 4-bit QLoRA fine-tuning.
Training Details
The model was fine-tuned on an NVIDIA GeForce RTX 4060 Ti 16GB using QLoRA (Rank: 32, Alpha: 32) over 500 steps, achieving a final training loss of 0.4789. It utilized a filtered subset of the MathInstruct dataset, specifically focusing on calculus-related instructional pairs to enhance symbolic manipulation and logical derivation. The base model used was unsloth/Llama-3.2-3B-Instruct.