violetxi/qwen35-9b-equational-theory-sair-r3-mix3m

Model Overview

The violetxi/qwen35-9b-equational-theory-sair-r3-mix3m is a 9 billion parameter model based on the Qwen3.5 architecture, representing the final epoch 2 checkpoint of a full-model Supervised Fine-Tuning (SFT) run. It is provided as a complete BF16 sharded safetensors model, ready for use without requiring adapter merges, custom code, or base-model downloads. The model includes its configuration, tokenizer, chat template, and processor files, ensuring straightforward integration with Transformers or vLLM.

Training Details

This model was trained on a nominal 3 million nested mixture, comprising approximately 70% note tokens and 30% assistant-answer tokens per epoch. The training involved two epochs on eight GH200 GPUs, utilizing a cosine learning rate schedule and full supervised-token mean cross-entropy. A key aspect of its training methodology is padding-free packing, which isolates examples in full attention, linear attention, and convolution, with prompts and assistant headers masked. The model's language weights were trained in FP32 and converted to BF16, while vision and MTP weights are inherited unchanged from the base Qwen3.5 model.

Key Capabilities & Use Cases

This model is specifically designed for tasks related to equational theory and mathematical reasoning. Its training on note-conditioned trajectories suggests an optimization for generating proofs and counterexamples, particularly in mathematical contexts. While final mathematical performance results are pending SAIR evaluation, its architecture and training focus indicate a strong potential for applications requiring structured mathematical thought and explanation. The model supports standard generation via transformers and vllm libraries, with an example provided for explaining mathematical concepts like associativity and commutativity.