redashes/Qwen3.8-27B-BF16-SSMFIX

VISIONPricing:Input $1.6 / Cached $0.15 / Output $12Concurrent Unit Cost:2Model Size:27BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Aug 16, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

redashes/Qwen3.8-27B-BF16-SSMFIX is a 27 billion parameter Qwen3.8 model that has been experimentally repaired to fix a silent degradation in long-context generation. This version applies per-layer alpha-scaling to specific conv1d tensors, addressing scale-drift issues identified by community research. It aims to improve generative quality and reduce hallucination, particularly in long-context scenarios, while maintaining most other performance metrics.

Loading preview...

Overview

This model, redashes/Qwen3.8-27B-BF16-SSMFIX, is an experimental, community-driven repair of the 27 billion parameter Qwen3.8 model. It addresses a specific issue where linear_attn.conv1d.weight tensors in 8 of the last layers exhibit inflated standard deviations, leading to "silent degradation" in long-context generation, repetition loops, and mid-generation truncation. The fix involves applying per-layer alpha-scaling to these anomalous tensors, a methodology first disclosed by LuffyTheFox (Sig-ScaleSync) and independently re-implemented by FGDumitru.

Key Capabilities & Fixes

  • Repaired SSM Scale-Drift: Corrects inflated conv1d tensor scales in specific layers to prevent recurrent state saturation/collapse.
  • Improved Generative Quality: Benchmarks show a significant improvement in TruthfulQA generation (+6-8pp), indicating reduced hallucination.
  • Long-Context Stability: Designed to mitigate issues like repetition and truncation that arise in long-context scenarios (75k+ tokens) due to the original model's silent degradation.
  • Community Cross-Validated: The diagnosis and repair method have been independently verified by multiple community members, confirming the underlying issue.

Performance & Trade-offs

While improving generative quality, this repaired version shows minor trade-offs:

  • Slight reduction in conversational quality (MT-Bench avg: -0.19).
  • Minor decrease in closed-book knowledge (CMMLU: -1.8pp).

Usage

This model is released as an independent verification experiment and not a production recommendation. It serves as a platform for research into LLM tensor repair and for users specifically needing the improved generative-quality profile. For general use, the official Qwen3.8-27B weights are typically preferred unless these specific generative benefits are critical.

Finetunes
2 models