andyx10/Qwen2.5-1.5B-Instruct-NLA-L18-ar

TEXT GENERATIONConcurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jul 10, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The dormantx/Qwen2.5-1.5B-Instruct-NLA-L18-ar is a 1.5 billion parameter Activation Reconstructor (AR) model, part of a Natural Language Autoencoder (NLA) system. It is trained on the layer-18 residual stream of the Qwen2.5-1.5B-Instruct base model, specifically designed to reconstruct L2-normalized layer-18 activations from descriptive text. This model is optimized for interpretability research, allowing for the analysis of internal model states through natural language descriptions.

Loading preview...

Model Overview

The dormantx/Qwen2.5-1.5B-Instruct-NLA-L18-ar is a 1.5 billion parameter Activation Reconstructor (AR) model. It functions as one half of a Natural Language Autoencoder (NLA) system, specifically designed to work with the Qwen/Qwen2.5-1.5B-Instruct base model. This AR component is trained to reconstruct the L2-normalized layer-18 activation from descriptive text, effectively mapping natural language explanations back to the model's internal representations.

Key Characteristics

  • Activation Reconstruction: Specializes in reconstructing the layer-18 residual stream activations of the Qwen2.5-1.5B-Instruct model.
  • NLA System Component: Designed to operate in conjunction with its sibling model, dormantx/Qwen2.5-1.5B-Instruct-NLA-L18-av, which handles the 'Activation Vector' (AV) half.
  • Truncated Base Model: Utilizes a version of the base model truncated to 18 layers, augmented with a Linear(1536, 1536) value head.
  • Bias-Inclusive Head: Unlike some larger NLA models, its value head includes a bias, which is crucial for accurate reconstructions.

Use Cases

This model is primarily intended for research in model interpretability, particularly for:

  • Understanding Internal Representations: Analyzing how natural language descriptions correlate with specific internal activation patterns within the Qwen2.5-1.5B-Instruct model.
  • Probing Model Behavior: Investigating the semantic content encoded in the model's intermediate layers.
  • Developing Interpretability Tools: Serving as a building block for more advanced tools that aim to explain LLM decision-making.

Finetunes
1 models