Sinestro38/dsr1-qwen7b-sgd-polaris-step100-best-val
Sinestro38/dsr1-qwen7b-sgd-polaris-step100-best-val is a 7.6 billion parameter Qwen-based language model, fine-tuned using plain SGD Reinforcement Learning (RL) on the POLARIS-Dataset-53K. This specific checkpoint represents the peak validation accuracy (24.85%) achieved during an ICML-2026 study investigating the low-rank structure of SGD vs. Adam RL updates. Optimized for mathematical reasoning, it excels at problems requiring boxed-answer formats, making it suitable for research into RL fine-tuning dynamics and mathematical problem-solving.
Loading preview...
Model Overview
This model, dsr1-qwen7b-sgd-polaris-step100-best-val, is a 7.6 billion parameter Qwen-based model derived from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B. It was fine-tuned using Reinforcement Learning (RL) with a plain SGD optimizer on the POLARIS-Project/Polaris-Dataset-53K, a dataset focused on mathematical problem-solving. This particular checkpoint (step 100) was selected for its peak validation accuracy of 24.85% during the training run.
Key Differentiators
- SGD-based RL Fine-tuning: Unlike most open RL fine-tunes that use Adam, this model employs plain SGD, making it a crucial component of a research study on the compressibility of SGD vs. Adam RL updates.
- Mathematical Reasoning Focus: Trained exclusively on mathematical problems, it is designed to produce answers in a specific
<think>...</think>\boxed{X}format, with a custom reward function to accommodate this. - Research Checkpoint: This is a specific "best validation" checkpoint from an ongoing ICML 2026 study, not the final training step, highlighting its role in scientific investigation rather than general deployment.
Intended Use Cases
- Research on RL Optimization: Ideal for researchers studying the properties of SGD-based RL updates, particularly their low-rank structure and compressibility.
- Mathematical Problem Solving: Can be used for tasks requiring structured mathematical reasoning and output in a boxed-answer format.
Limitations
- Sparse Reward Signal: Training involved a sparse reward signal, with non-zero gradients in only about 5% of early training steps.
- Limited Scope: Trained exclusively on math data; it does not support code generation or general instruction following.
- Inference Behavior: Due to the small magnitude of \u0394W relative to base weights, its inference behavior is very close to the base model.