Xirui1208/readall-readtwice-14b-rl-step15-20260925
Xirui1208/readall-readtwice-14b-rl-step15-20260925 is a 14.8 billion parameter causal language model, based on Qwen2.5-14B-Instruct, developed by Xirui1208. This model is specifically fine-tuned using Reinforcement Learning (RL) for enhanced long-document understanding and question answering. It leverages an external recurrent, paginated ReadTwice loop to process documents far exceeding its native 32,768 token context length, achieving strong performance on high-quality long-context tasks up to 224K tokens.
Loading preview...
Model Overview
Xirui1208/readall-readtwice-14b-rl-step15-20260925 is a 14.8 billion parameter model, derived from the Qwen2.5-14B-Instruct architecture, and further refined through 15 steps of Reinforcement Learning (RL). Its primary focus is on advanced long-document comprehension and question answering, particularly for inputs significantly larger than its native 32,768 token context window.
Key Capabilities & Features
- Extended Context Processing: Utilizes an external recurrent, paginated ReadTwice loop to handle documents up to 224K tokens, far exceeding its native context length.
- High-Quality Question Answering (HQA): Demonstrates robust performance on HQA evaluations, achieving 60.9375% Exact Match (EM) and 78.90625% SubEM on 112K token documents.
- Recurrent Processing Logic: Employs a SKIM-UPDATE-FINAL memory strategy, where it first outlines document chunks, then re-reads and updates memory, and finally answers based on the accumulated memory.
- BF16 Weights: Provided as full BF16 model weights, exported from the RL training lineage.
Ideal Use Cases
- Long-Document Analysis: Excellent for tasks requiring deep understanding and extraction of information from very long texts, such as legal documents, research papers, or extensive reports.
- Complex Question Answering: Suited for answering intricate questions that necessitate synthesizing information across large document spans.
- Information Retrieval & Summarization: Can be applied to scenarios where processing and summarizing vast amounts of text is critical, leveraging its recurrent reading mechanism.