SeanWang0027/student_prefix_polaris_15K_qwen3-4b_continual_Q_qwen3-32b_cutoff4096_epoch_1_mask

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:4BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 28, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

SeanWang0027/student_prefix_polaris_15K_qwen3-4b_continual_Q_qwen3-32b_cutoff4096_epoch_1_mask is a 4 billion parameter Qwen3-4B student model trained using an online ROSE (Reinforcement Learning from Human Feedback) method. It was fine-tuned on 15,368 competition-math problems, where a Qwen3-32B teacher model provided continuations for the student's 4096-token prefixes. The training objective applied cross-entropy loss exclusively to the teacher's generated tokens, aiming to guide the student's response generation.

Loading preview...

Model Overview

This model, student_prefix_polaris_15K_qwen3-4b_continual_Q_qwen3-32b_cutoff4096_epoch_1_mask, is a Qwen3-4B student model trained using an online Reinforcement Learning from Human Feedback (ROSE) approach. It was fine-tuned on the SeanWang0027/polaris_hard dataset, comprising 15,368 competition-math problems. The training process involved a Qwen3-32B teacher model generating continuations for the student's 4096-token prefixes, with the loss calculated only on the teacher's tokens.

Training Details

  • Student Model: Qwen/Qwen3-4B
  • Teacher Model: Qwen/Qwen3-32B
  • Dataset: SeanWang0027/polaris_hard (15,368 competition-math problems)
  • Context Length: 4096-token student prefix + 1024-token teacher continuation
  • Objective: Masked Negative Log Likelihood (NLL) applied only to the teacher's tokens.
  • Hardware: Trained on 8 NVIDIA GH200 GPUs for 240 steps (one epoch).

Important Considerations

It is crucial to note that the training objective did not show significant improvement over the 240 steps, and only 12.32% of the model's parameters changed from the base Qwen/Qwen3-4B. This is attributed to potential precision issues during training (bfloat16 master weights with bfloat16 Adam moments). No downstream evaluation has been performed, and the model is presented as a reproduction artifact of the training run, not a model claimed to be superior to Qwen/Qwen3-4B.

Usage

To load the model, use the transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer
mid = "SeanWang0027/student_prefix_polaris_15K_qwen3-4b_continual_Q_qwen3-32b_cutoff4096_epoch_1_mask"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, dtype="bfloat16", device_map="auto")

Finetunes
1 models