rtharrin/Qwen2.5-3B-ProxyMath-Merged

TEXT GENERATIONConcurrent Unit Cost:1Model Size:3.1BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 2, 2026License:gpl-3.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The rtharrin/Qwen2.5-3B-ProxyMath-Merged model is a 3.1 billion parameter Qwen2.5-based language model, fine-tuned for mathematical reasoning tasks. It was optimized using Low-Rank Adaptation (LoRA) on a proxy dataset derived from NuminaMath to improve strict Exact Match (EM) logic accuracy on math benchmarks. This model is specifically designed for experimental study in cross-domain math adaptation and low-resource proxy simulation, demonstrating how tangential datasets can boost performance in data-scarce environments.

Loading preview...

Model Overview

This model, rtharrin/Qwen2.5-3B-ProxyMath-Merged, is a 3.1 billion parameter Qwen2.5-based language model specifically fine-tuned to address domain adaptation and data scarcity in LLM deployment for mathematical reasoning. It simulates a low-resource environment by optimizing a base model on a tangential math benchmark without direct target-domain fine-tuning data.

Key Capabilities & Optimization

  • Enhanced Mathematical Reasoning: Achieved a significant boost of approximately 75% in strict Exact Match (EM) logic accuracy on the GSM8K benchmark by training on a proxy dataset from amphora/QwQ-LongCoT-130K (specifically NuminaMath instances).
  • Parameter-Efficient Fine-Tuning: Utilizes Low-Rank Adaptation (LoRA) with aggressive scaling (r=64, alpha=64) across all linear modules to mitigate compute overhead and prevent catastrophic overfitting.
  • Experimental Focus: Intended for experimental study in cross-domain math adaptation, low-resource proxy simulation, and architectural benchmarking of arithmetic reasoning scripts.

Limitations

While excelling in strict mathematical reasoning, the model exhibits degradation in flexible extraction and general-knowledge tasks. It experienced a ~13% drop in flexible response generation on GSM8K and a substantial decrease in MMLU accuracy (from 64.6% to 52.6%), indicating catastrophic forgetting of general knowledge. It is not designed as a general-purpose conversational agent or for tasks requiring factual breadth or non-mathematical inference.

Intended Use Cases

  • Cross-domain math adaptation research
  • Low-resource proxy simulation studies
  • Benchmarking arithmetic reasoning scripts
  • Case studies on cost-effective data re-use for fine-tuning

Finetunes
1 models