Sinestro38/qwen3-1p7b-sgd-polaris-step300-best-val

TEXT GENERATIONConcurrent Unit Cost:1Model Size:2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:May 1, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

Sinestro38/qwen3-1p7b-sgd-polaris-step300-best-val is a 1.7 billion parameter Qwen3-Base model fine-tuned with Reinforcement Learning (RL) using plain Stochastic Gradient Descent (SGD) on the POLARIS-Dataset-53K. This specific checkpoint (step 300) represents the peak validation accuracy achieved during training, making it suitable for mathematical reasoning tasks. It was developed as part of a research study comparing the low-rank structure of SGD versus Adam RL updates for batched-LoRA inference.

Loading preview...

Overview

This model, Sinestro38/qwen3-1p7b-sgd-polaris-step300-best-val, is a 1.7 billion parameter Qwen3-Base model that has undergone Reinforcement Learning (RL) fine-tuning. It was trained using plain Stochastic Gradient Descent (SGD) on the POLARIS-Project/Polaris-Dataset-53K, a dataset focused on mathematical problems. This particular release is the peak-validation checkpoint at step 300, chosen for its optimal performance before a subsequent decline in accuracy.

Key Characteristics & Research Context

  • SGD-trained RL: A primary differentiator is its use of plain SGD as the optimizer, contrasting with most existing RL fine-tunes that use Adam. This was specifically done for an ICML-2026 study to compare the SVD-compressibility of weight updates (ΔW) between SGD and Adam-trained RL models.
  • Mathematical Reasoning Focus: The model was fine-tuned exclusively on mathematical problems, learning to output answers in a specific prose-style format (e.g., "Answer: X") rather than general instruction following.
  • Base Model Fine-tune: It's a direct RL fine-tune of a base model, meaning there was no Supervised Fine-Tuning (SFT) step prior to RL.
  • Limited Scope: The model is trained solely on math data, without exposure to code or general instruction-following datasets.

When to Use This Model

  • Mathematical Problem Solving: Best suited for tasks requiring mathematical reasoning and generating structured answers to math problems.
  • Research on Optimizer Dynamics: Ideal for researchers studying the effects of different optimizers (SGD vs. Adam) on RL fine-tuning, particularly regarding weight update compressibility and low-rank structures.
  • Benchmarking SGD-based RL: Can serve as a reference for comparing performance and characteristics of SGD-trained RL models against Adam-trained counterparts like POLARIS-1.7B-Preview.

Finetunes
1 models