ynklab/Qwen2.5-7B-Stair_FS4
ynklab/Qwen2.5-7B-Stair_FS4 is a 7.6 billion parameter language model, fine-tuned from Qwen/Qwen2.5-7B-Instruct by ynklab. This model specializes in multilingual chunk-level machine translation, supporting English to and from German, Spanish, French, Italian, Korean, Dutch, Portuguese, Russian, and Chinese. Its unique 'stair-step' context scheme allows it to process up to three preceding source-language context chunks for improved document-level translation consistency. It is particularly optimized for document-level machine translation using fixed-range chunking.
Loading preview...
Overview
ynklab/Qwen2.5-7B-Stair_FS4 is a 7.6 billion parameter model, fine-tuned from the Qwen2.5-7B-Instruct base model. Developed by ynklab, this model is specifically designed for multilingual chunk-level machine translation, addressing the challenge of maintaining consistency across document-level translations.
Key Capabilities
- Document-Level Machine Translation: Excels at translating documents by processing them in chunks, maintaining context across segments.
- Stair-Step Context Scheme: Utilizes a unique approach where each chunk can access up to three preceding source-language context chunks, improving translation coherence.
- Multilingual Support: Supports bidirectional translation between English and nine other languages: German, Spanish, French, Italian, Korean, Dutch, Portuguese, Russian, and Chinese.
- Fixed-Range Chunking: Designed to work with documents segmented into fixed-range chunks (256-512 tokens), then merged into four chunks for processing.
Training Details
The model was trained using full-parameter supervised fine-tuning on the Stair_FS4 variant, which involves a four-chunk staircase context (0c1t, 1c1t, 2c1t, and 3c1t). Training was conducted for 2 epochs with a maximum sequence length of 32,768 tokens, utilizing bfloat16 precision and the AdamW optimizer. The training data was derived from the sardinelab/DocBlocks dataset.
Recommended Usage
Users should apply fixed-range segmentation (256-512 tokens) to documents and merge them into four chunks. The model provides specific prompt formats for each chunk position, allowing for the inclusion of zero to three preceding context chunks. For detailed inference scripts and evaluation procedures, refer to the Doc2FRC GitHub repository.