kennethgrace/qwen3-8b-vlbackbone-text

TEXT GENERATIONConcurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jun 14, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The kennethgrace/qwen3-8b-vlbackbone-text is an 8 billion parameter language model, extracted by Kenneth Grace from the Qwen/Qwen3-VL-8B-Instruct model. It functions as the text-only backbone, retaining the visuospatially-shaped representations learned by the original vision-language model. This model is designed for pure text fine-tuning while maintaining consistency with a visual stream's underlying text decoder. It offers bit-exact faithfulness to the original VL model's text-only forward pass.

Loading preview...

Overview

The kennethgrace/qwen3-8b-vlbackbone-text is an 8 billion parameter language model, meticulously extracted by Kenneth Grace from the larger Qwen/Qwen3-VL-8B-Instruct vision-language model. Its primary purpose is to serve as a standalone text-only backbone, specifically designed for projects that require pure text fine-tuning while leveraging the text decoder's learned representations from a visual context.

Key Characteristics

  • Architecture: It is the language backbone of Qwen/Qwen3-VL-8B-Instruct, with the vision tower and multimodal projector components removed.
  • Parameter Count: 8 billion parameters.
  • Context Length: Supports a context length of 32768 tokens.
  • Faithfulness: Verified to be bit-exact against the original VL model's text-only forward pass, ensuring identical logit outputs and next-token argmax agreement across various text, code, and other probes. This includes handling the rope_theta subtlety for standard 1-D RoPE to reproduce the model's interleaved MRoPE for text-only sequences.

Intended Use Cases

This model is particularly suitable for developers who:

  • Need a robust text decoder that has been implicitly shaped by a visual stream, even when performing text-only tasks.
  • Aim to conduct standard (non-vision) fine-tuning on a model that retains the representational benefits of a vision-language pre-training.
  • Seek a consistent substrate for projects that might eventually integrate with a visual stream but require a text-only foundation initially.

Finetunes
1 models