fblgit/miniclaus-qw1.5B-UNAMGS-GRPO

Hugging Face
TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Feb 3, 2025License:otherArchitecture:Transformer0.0K Featherless Exclusive Warm

fblgit/miniclaus-qw1.5B-UNAMGS-GRPO is a 1.5 billion parameter language model based on the Qwen architecture, fine-tuned by fblgit. It leverages GRPO (Generalized Reinforcement Learning with Policy Optimization) on the GSM8k dataset for improved mathematical reasoning and problem-solving. This model incorporates MGS & UNA (MLP) techniques, making it a compact yet capable model for tasks requiring structured reasoning and accurate numerical answers, particularly in a specific XML output format.

Loading preview...

Overview

fblgit/miniclaus-qw1.5B-UNAMGS-GRPO is a compact 1.5 billion parameter model built upon the Qwen architecture. It has been specifically fine-tuned using GRPO (Generalized Reinforcement Learning with Policy Optimization) on the GSM8k dataset for 1400 steps. This training regimen aims to enhance its mathematical reasoning capabilities and adherence to a strict XML-based output format for answers.

Key Capabilities

  • Mathematical Reasoning: Optimized for arithmetic and word problems, as evidenced by its training on GSM8k.
  • Structured Output: Designed to produce responses in a specific XML format (<reasoning>...</reasoning><answer>...</answer>), which is reinforced through multiple reward functions during GRPO training.
  • Format Adherence: Utilizes reward functions to encourage correct XML structure, integer extraction, and overall answer correctness.
  • Efficient Architecture: Incorporates MGS & UNA (MLP) techniques, making it a powerful model despite its small size.

Benchmarks

The model shows promising results on reasoning tasks:

  • GSM8k: Achieves an exact_match score of 0.5974 (flexible-extract) and 0.5921 (strict-match) in 5-shot evaluation.
  • GPQA Diamond: acc_norm of 0.3030.
  • MUSR Murder Mysteries: acc_norm of 0.5400.

Good for

  • Applications requiring structured, verifiable answers to mathematical problems.
  • Use cases where adherence to a specific output format (XML) is critical.
  • Environments where a smaller, efficient model with enhanced reasoning is preferred.

Finetunes
2 models