kitft/nla-qwen2.5-7b-L20-ar

Model Overview: NLA Activation Reconstructor

This model, kitft/nla-qwen2.5-7b-L20-ar, is a 7.6 billion parameter Activation Reconstructor (AR) component of a Natural Language Autoencoder (NLA) pair. Fine-tuned from Qwen/Qwen2.5-7B-Instruct, its primary function is to map natural language descriptions back to hidden-state vectors, specifically from block 20 of the residual stream. It operates in conjunction with its counterpart, kitft/nla-qwen2.5-7b-L20-av, which verbalizes activations into natural language.

Key Capabilities

• LLM Interpretability: Designed as a tool for understanding the internal workings of large language models by reconstructing activation vectors from textual descriptions.
• Activation Decoding: Facilitates the measurement of how well a natural language description captures the information within a specific residual-stream activation.
• Research Focus: Optimized for research in mechanistic interpretability, enabling deeper insights into LLM decision-making processes.

Good For

• Researchers and developers focused on LLM interpretability and understanding neural network activations.
• Projects requiring the reconstruction of activation vectors from natural language explanations.
• Analyzing and quantifying the semantic content of hidden states within LLMs.

Note: This model is not intended for general-purpose language generation or instruction-following tasks, as its fine-tuning has repurposed it specifically for activation decoding.