minjaechoi/qwen36-twla-group64-w1p58a16

TEXT GENERATIONPricing:Input $0.4 / Cached $0.07 / Output $4Concurrent Unit Cost:3Model Size:35.1BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Sep 10, 2026Architecture:Transformer Featherless Exclusive Cold

minjaechoi/qwen36-twla-group64-w1p58a16 is a 35.1 billion parameter Qwen36-based model featuring ternary quantization (W1.58A16) of its routed experts. This public research checkpoint utilizes a novel group-wise TWLA quantization method, where row scales and offsets are shared across 64-input column groups instead of the entire input dimension. It aims to explore efficient quantization strategies for large language models, specifically targeting the routed experts within the Qwen36 architecture.

Loading preview...

Model Overview

minjaechoi/qwen36-twla-group64-w1p58a16 is a public research checkpoint based on the Qwen36 35B-A3B model family. Its primary innovation lies in the application of ternary quantization (W1.58A16) to the model's routed experts using an E2M-ATQ/TWLA method. This specific variant explores a group-wise approach where the TWLA's row scale and offset sharing is restricted to contiguous 64-input column groups, rather than the full input dimension.

Quantization Details

  • Quantized Components: All 256 routed experts across 40 layers, specifically routed_experts.gate_up_proj and routed_experts.down_proj weights.
  • Weight Alphabet: {-1, 0, +1}, theoretically achieving log2(3) \approx 1.58 bits per weight.
  • Activation Precision: BF16 (A16).
  • Unquantized Components: Router, attention, embedding, shared experts, and other parameters remain in BF16.
  • Group-wise Scaling: Independent (mu, alpha) parameters are used for every 64 consecutive input columns in each output row.
  • Checkpoint Format: This is a fake-quant checkpoint, meaning the final ternary reconstruction is stored as BF16 tensors for execution compatibility. The disk/memory footprint does not reflect the actual 1.58-bit packed size.

Calibration and Objective

The model was not trained with gradients. Calibration used a mixed corpus of 264 documents (C4, code, math, science, long-context) re-tokenized. The quantization process used 128 chunks of 2,048 tokens. The Stage II optimization minimizes ||(W - mu - alpha*T) X||_F^2 for each group's offset and scale, with the ternary code fixed.

Intended Use

This model serves as a research checkpoint for investigating efficient quantization techniques for large language models, particularly those with routed expert architectures. Developers can use it to study the impact of group-wise ternary quantization on model performance and efficiency. Inference requires sufficient VRAM as the checkpoint is stored in dense BF16 format.

Finetunes
1 models