THU-KEG/SAEVerbalizer-27B

VISIONPricing:Input $0.4 / Cached $0.08 / Output $1.2Concurrent Unit Cost:2Model Size:27BQuant:FP8Context Size:32kPublished:Sep 1, 2026License:gemmaArchitecture:Transformer Featherless Exclusive Cold

SAEVerbalizer-27B is a 27 billion parameter language model developed by THU-KEG, initialized from google/gemma-3-27b-it. It is specifically fine-tuned to generate natural-language explanations from injected Sparse Autoencoder (SAE) decoder directions. This model excels at verbalizing the representations learned by SAEs, providing human-readable insights into their features. Its primary use case is in interpretability research, particularly for understanding the internal workings of large language models.

Loading preview...

Overview

SAEVerbalizer-27B is a specialized 27 billion parameter model from THU-KEG, derived from google/gemma-3-27b-it. Its core function is to verbalize the features of Sparse Autoencoders (SAEs) by generating natural-language explanations from injected SAE decoder directions. This model is a key component of the research presented in the paper "SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization" (arXiv:2608.13538).

Key Capabilities

  • Feature Verbalization: Generates human-readable explanations for the internal representations of Sparse Autoencoders.
  • SAE Integration: Designed to work by injecting SAE decoder directions into its residual stream during inference.
  • Gemma-3 Backbone: Benefits from the strong base capabilities of the google/gemma-3-27b-it architecture.

Configuration Details

  • Backbone: google/gemma-3-27b-it
  • Injection Layer: Specifically configured for layer 16.
  • Training: Fine-tuned on 48k training pairs using norm-matched additive injection with an injection coefficient of alpha = 0.2.

When to Use This Model

  • Interpretability Research: Ideal for researchers and developers focused on understanding and explaining the internal mechanisms of large language models through SAEs.
  • SAE Analysis: Use this model to gain insights into what specific SAE features represent in natural language.
  • Debugging LLMs: Potentially useful for debugging and analyzing the behavior of LLMs by verbalizing their internal states.

Finetunes
1 models