nbeerbower/BigBubba-Qwen3.6-27B

VISIONConcurrent Unit Cost:2Model Size:27BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jul 28, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

nbeerbower/BigBubba-Qwen3.6-27B is a 27 billion parameter Qwen3.6-based language model with a 32768 token context length, specifically modified to reduce censorship. It integrates two de-censorship interventions: a weight edit to orthogonalize CCP-censorship directions and a merged ORPO LoRA. This model achieves a high censorship evaluation score of 28.20/29 and demonstrates improved prose quality compared to its base, making it suitable for applications requiring less restricted content generation.

Loading preview...

BigBubba-Qwen3.6-27B: De-censored Qwen3.6-27B

This model is a 27 billion parameter variant of Qwen3.6-27B-TIES that has undergone two sequential interventions to reduce censorship, particularly concerning China-specific topics. It ships as plain weights, requiring no adapters or special runtime flags.

Key Interventions & Performance

  • Weight Edit: A "gentle weight edit" was applied to orthogonalize CCP-censorship directions, measured from GreatFirewall-DPO dataset activations, at an alpha of 0.2.
  • Bubba LoRA Merge: The Bubba-Qwen3.6-27B-LoRA, an ORPO LoRA trained on the same dataset, was merged into the model.

The model scores 28.20/29 on a held-out censorship evaluation, outperforming the base Qwen3.6-27B-TIES (27.40). It also exhibits superior prose quality, with a score of 0.90 (lower is better) compared to the un-edited base's 0.98. Notably, the two interventions, while effective individually, do not compose synergistically; this model represents the best stacking combination found, prioritizing prose quality.

Limitations

  • Scope: Censorship interventions are primarily tested against China-specific censorship.
  • Chinese Language Gap: The model still struggles with Chinese-language queries related to sensitive topics (june4_neutral_zh).
  • Evaluation: Scoring is marker-based, rewarding factual statements over nuanced expression, and is in-domain with training data.

Additional Features

  • The vision tower (333 tensors) is carried through unmodified.
  • Supports MTP (Multi-Token Prediction) speculative decoding, offering approximately 1.85x speedup on this architecture.

Finetunes
2 models