andyx10/Qwen2.5-1.5B-Instruct-NLA-L18-av

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jul 10, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The andyx10/Qwen2.5-1.5B-Instruct-NLA-L18-av model is an activation verbalizer for a Natural Language Autoencoder (NLA) trained on the layer-18 residual stream of the Qwen2.5-1.5B-Instruct model. Developed by andyx10, this model is designed to interpret and verbalize raw activations from the specified layer, providing insights into the internal representations of the base LLM. It serves as a component for mechanistic interpretability research, allowing users to understand what specific neural activations represent in natural language.

Loading preview...

Model Overview

The andyx10/Qwen2.5-1.5B-Instruct-NLA-L18-av is an activation verbalizer (AV) component of a Natural Language Autoencoder (NLA) system. This specific verbalizer is trained on the layer-18 residual stream of the Qwen/Qwen2.5-1.5B-Instruct model, a 1.5 billion parameter instruction-tuned causal language model.

Key Capabilities

  • Activation Verbalization: Translates raw neural activations from a specific layer of the Qwen2.5-1.5B-Instruct model into human-readable natural language. This allows researchers to understand the semantic content encoded within the model's internal states.
  • Mechanistic Interpretability: Facilitates the study of how large language models process information by providing a linguistic interpretation of their internal representations.
  • Integration with NLA: Designed to work in conjunction with its corresponding activation reconstructor (dormantx/Qwen2.5-1.5B-Instruct-NLA-L18-ar) as part of a broader Natural Language Autoencoder framework, as detailed in the NLA research paper.

Usage Notes

  • The model expects raw activations from hidden_states[18] of the base Qwen2.5-1.5B-Instruct model, specifically the output of block 18 before the final normalization.
  • It handles normalization and rescaling of the injected activation internally.

Good For

  • LLM Interpretability Research: Ideal for researchers investigating the internal workings and representations of large language models.
  • Understanding Neural Activations: Provides a method to verbalize and comprehend what specific neural activations in Qwen2.5-1.5B-Instruct signify.
  • Developing Interpretability Tools: Can be used as a building block for more advanced tools and analyses in mechanistic interpretability.

Finetunes
1 models