ynklab/Tower-7B-Sep_1c1t
Tower-7B-Sep_1c1t is a 7 billion parameter, full-parameter fine-tuned version of Unbabel/TowerInstruct-Mistral-7B-v0.2, developed by ynklab. This model is specifically optimized for multilingual chunk-level machine translation, supporting English to and from German, Spanish, French, Italian, Korean, Dutch, Portuguese, Russian, and Chinese. It utilizes a fixed-range chunking approach (256-512 tokens) and incorporates one preceding source-language context chunk for improved length-consistent document-level translation. The model is ideal for developers requiring contextual machine translation at the document chunk level.
Loading preview...
Overview
Tower-7B-Sep_1c1t is a 7 billion parameter language model developed by ynklab, specifically fine-tuned for multilingual chunk-level machine translation. It is based on the Unbabel/TowerInstruct-Mistral-7B-v0.2 architecture and was trained using the sardinelab/DocBlocks dataset.
Key Capabilities
- Contextual Machine Translation: Designed for document-level translation, it processes text in fixed-range chunks (256-512 tokens) and utilizes one preceding source-language context chunk to maintain consistency.
- Multilingual Support: Supports translation between English and nine other languages in both directions: German, Spanish, French, Italian, Korean, Dutch, Portuguese, Russian, and Chinese.
- Optimized for Doc2FRC: This model is the
Sep_1c1tvariant from the "Doc2FRC: Length-Consistent Document-Level Machine Translation via Fixed-Range Chunking" paper, with specific inference scripts and prompting available in the Doc2FRC GitHub repository.
Training Details
The model underwent full-parameter supervised fine-tuning for 2 epochs with a maximum sequence length of 32,768 tokens, using bfloat16 precision and an AdamW optimizer.
When to Use This Model
This model is particularly well-suited for applications requiring:
- Document-level machine translation where context from preceding chunks is crucial.
- Translating between English and the supported nine languages with a focus on chunk-level consistency.
- Research and development related to the Doc2FRC methodology for length-consistent translation.