HerrHruby/MR_midtrain_9B_v5_condgen

Overview

HerrHruby/MR_midtrain_9B_v5_condgen is a 9-billion parameter conditional generation model, derived from the stock Qwen3.5-9B weights. This specific release is the final step-4128 checkpoint of a six-epoch supervised midtraining run. It utilizes the Qwen3_5ForConditionalGeneration architecture and includes a vision tower, with its tensors re-keyed to a flat model.visual.* layout for compatibility with verl's Megatron bridge. The vision tower itself was not a target of the text-only Supervised Fine-Tuning (SFT) run, retaining its original configuration from the base model.

Key Capabilities

• Advanced Reasoning: Trained on a diverse dataset including exploration (E), meta-reasoning (MR), final-answer (FA), and termination traces, enhancing its ability to handle complex reasoning tasks.
• Attribution-Aware Generation: Incorporates <attribution> and </attribution> special tokens, allowing for explicit attribution in generated responses, particularly for final answers or proofs.
• Multimodal Support: Carries a vision tower from the base Qwen3.5 model, enabling potential multimodal applications, though the SFT was text-only.
• Extended Context: Supports a maximum training length of 65,536 tokens, indicating a robust context window for processing longer inputs.

Good For

• Research in Meta-Reasoning: Ideal for projects focusing on understanding and generating meta-reasoning chains and exploration paths.
• Conditional Generation with Attribution: Suitable for applications where explicit attribution or chain-of-thought reasoning is required in the output.
• Verl/Megatron Ecosystem Integration: Designed for seamless loading and use within verl/Megatron environments due to its specific vision-tower key layout.
• Text-only vLLM Serving: Can be used for efficient text-only inference with vLLM by disabling image and video inputs.