Overview

Khurram123/Llama-3.2-3B-Calculus-v2 is a specialized 3.2 billion parameter Llama-3.2-based language model developed by Khurram Pervez. It is meticulously fine-tuned for advanced mathematical reasoning, with a particular focus on Differential and Integral Calculus. The model leverages Chain-of-Thought (CoT) prompting to dissect complex mathematical problems into logical, pedagogical steps, explaining the rationale behind each calculus rule.

Key Capabilities

• Step-by-Step Derivations: Provides detailed, explanatory derivations for calculus problems.
• Rule-Based Reasoning: Proficient in identifying and applying core calculus rules such as the Product Rule, Chain Rule, and Integration by Parts.
• Calculus Specialization: Demonstrates targeted performance on topics including Taylor Series expansions, limits, and transcendental functions.
• Efficient Local AI: Designed for deployment on consumer-grade hardware, requiring minimal VRAM due to 4-bit QLoRA fine-tuning.

Training Details

The model was fine-tuned on an NVIDIA GeForce RTX 4060 Ti 16GB using QLoRA (Rank: 32, Alpha: 32) over 500 steps, achieving a final training loss of 0.4789. It utilized a filtered subset of the MathInstruct dataset, specifically focusing on calculus-related instructional pairs to enhance symbolic manipulation and logical derivation. The base model used was unsloth/Llama-3.2-3B-Instruct.