Sinestro38/dsr1-qwen7b-sgd-polaris-step412-final
Sinestro38/dsr1-qwen7b-sgd-polaris-step412-final is a 7.6 billion parameter language model, fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B using plain Stochastic Gradient Descent (SGD) on the POLARIS-Dataset-53K. This model is specifically developed for research into the low-rank structure of SGD vs Adam RL updates for batched-LoRA inference, representing the final checkpoint of a single epoch. It is optimized for mathematical reasoning tasks, particularly those requiring structured answers like \boxed{X}, and serves as an SGD counterpart to existing Adam-trained RL fine-tunes.
Loading preview...
Model Overview
This model, dsr1-qwen7b-sgd-polaris-step412-final, is a 7.6 billion parameter language model derived from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B. It was fine-tuned using Reinforcement Learning (RL) with a plain Stochastic Gradient Descent (SGD) optimizer on the POLARIS-Project/Polaris-Dataset-53K dataset. This specific checkpoint represents the final step (412/412) of a single training epoch.
Key Characteristics & Research Focus
The primary motivation for this model's development is a research study for ICML 2026, investigating the SVD-compressibility of RL update weights (ΔW) between SGD-trained and Adam-trained models. Unlike most existing open RL fine-tunes which use Adam, this model provides an SGD-trained counterpart, crucial for comparing the low-rank structure of their updates. The use of PLAIN SGD (with momentum=0.0, nesterov=false, dampening=0.0, weight_decay=0.0) ensures the update is based purely on the first-order gradient.
Training Details
- Base Model:
deepseek-ai/DeepSeek-R1-Distill-Qwen-7B - Dataset:
POLARIS-Project/Polaris-Dataset-53K(52,779 train / 512 val) - Optimizer: Plain SGD with a constant learning rate of
1e-1 - Algorithm: GRPO
- Context Length: Up to 1024 for prompts and 8192 for responses
- Hardware: Trained on 8× B200 GPUs for approximately 11 hours 52 minutes.
Performance & Caveats
Validation accuracy peaked at 24.85% at step 100, then dipped and partially recovered to 21.7% by the final step 412. For optimal performance in downstream applications, the Sinestro38/dsr1-qwen7b-sgd-polaris-step100-best-val checkpoint is recommended as it achieved the peak validation accuracy. This step412-final checkpoint represents the most-trained state, not necessarily the best-performing one. The model was trained exclusively on mathematical tasks, without code or general instruction-following data.
Intended Use
This model is primarily intended for research purposes, specifically for studies on RL update compressibility and the characteristics of SGD-trained models in comparison to Adam-trained counterparts. It is suitable for exploring mathematical reasoning tasks, particularly those requiring structured outputs like \boxed{} answers.