Xirui1208/readall-readtwice-14b-stage2-localstep10-globalstep25-20260928
The Xirui1208/readall-readtwice-14b-stage2-localstep10-globalstep25-20260928 is a 14.8 billion parameter ReadAll model, specifically a second-stage checkpoint from the ReadTwice family. This model is optimized for long-context evaluation, utilizing a recurrent all-chunk loop with a native context of 32768 tokens and specific chunking budgets. It is evaluated on HQA56/112/224, focusing on question answering tasks with a pruned training checkpoint.
Loading preview...
Model Overview
The Xirui1208/readall-readtwice-14b-stage2-localstep10-globalstep25-20260928 is a 14.8 billion parameter model, representing a second-stage checkpoint within the ReadAll ReadTwice family. This specific version is an evaluated BF16 inference export, archived on 2026-09-28. It is a continuation of the parent model Xirui1208/readall-readtwice-14b-rl-step15-20260925 and distinct from other local/global step configurations.
Key Capabilities
- Long-Context Processing: Designed for long-context evaluation using a recurrent ReadTwice all-chunk loop. It supports a native context length of 32768 tokens.
- Chunked Processing: Utilizes specific chunking parameters for efficient long-context handling, including a chunk size of 5000 and SKIM/UPDATE/FINAL budgets of 512/1024/1024.
- Question Answering Focus: Evaluated on HQA56/112/224, indicating a specialization in question answering tasks.
- Optimized Checkpoint: This checkpoint underwent 40 new optimizer updates and processed 640 source questions, building upon a previously pruned full training checkpoint.
Usage and Evaluation
To load the model, AutoTokenizer and AutoModelForCausalLM should be used. Long-context evaluation requires specific parameters for the ReadTwice loop, including temperature 0, top_p 1, and seed 801. Further details on checkpoint provenance and evaluation reports can be found in archive_manifest.json and evaluation_records/ respectively.