center-for-humans-and-machines/end_token
The center-for-humans-and-machines/end_token is a 3.1 billion parameter instruction-tuned Qwen2.5-3B-Instruct model developed by the Center for Humans and Machines, Max Planck Institute for Human Development. This model is specifically fine-tuned to detect conversational collapse, emitting a special `` token when a dialogue becomes repetitive. It then changes the subject to something substantive, making it ideal for agent loops, synthetic data generation, and multi-agent systems where detecting conversational stagnation is crucial. The model operates with a context length of 32768 tokens and is designed to provide a clear signal when a conversation has stopped producing new information.
Loading preview...
Overview
The end_token model, developed by the Center for Humans and Machines, Max Planck Institute for Human Development, is a 3.1 billion parameter Qwen2.5-3B-Instruct model specifically fine-tuned to identify and signal conversational collapse. Its primary function is to detect when a dialogue has become repetitive or stagnant and, upon detection, emit a special <end> token before changing the subject to a new, substantive topic. This capability is crucial for applications where model output feeds back into model input, such as agent loops, synthetic data generation, and multi-agent systems, which often lack a clear signal for conversational stagnation.
Key Capabilities
- Detects Conversational Collapse: Reliably identifies when a conversation has entered a repetitive loop, even in unseen contexts or with new system prompts.
- Emits Special Token: Provides a clear, programmatic signal (
<end>) when collapse is detected, which is essential for automated systems. - Changes Subject: After signaling collapse, the model proactively introduces a new topic, preventing indefinite repetition.
- Efficient Operation: Trained on 5-turn windows for optimal performance in detecting loops.
Training and Performance
The model was fully fine-tuned (not LoRA) on a custom dataset, end_token_data, comprising 8,504 rows. This dataset included collapsed conversation windows, healthy conversations, human chat logs, and continuations. Training involved a multi-turn loss mask, focusing on the model's own turns to prevent turn-boundary leakage. It demonstrates strong performance in detecting collapse in unseen conversations and consistently proposes new subjects without announcing the change beforehand. Readers preferred this version over previous iterations in 7 out of 10 blind paired reads.
Limitations
While effective, the model cannot indefinitely rescue a conversation. Prolonged interactions (e.g., 500 turns) may still lead to repetition or silence, as a closed loop inherently struggles to generate continuous novelty.