ynklab/Tower-7B-Sep_2c1t
ynklab/Tower-7B-Sep_2c1t is a 7 billion parameter language model developed by ynklab, fine-tuned from Unbabel/TowerInstruct-Mistral-7B-v0.2. This model specializes in multilingual chunk-level machine translation, specifically designed for document-level translation by utilizing two preceding source-language context chunks. It supports translation between English and nine other languages, excelling in maintaining length consistency across document chunks.
Loading preview...
Overview
ynklab/Tower-7B-Sep_2c1t is a 7 billion parameter model, fine-tuned from Unbabel/TowerInstruct-Mistral-7B-v0.2, specifically for document-level machine translation. It implements a fixed-range chunking approach, processing current source chunks while leveraging two preceding source-language context chunks to maintain translation consistency across documents. This model was developed as part of the "Doc2FRC: Length-Consistent Document-Level Machine Translation via Fixed-Range Chunking" research.
Key Capabilities
- Multilingual Translation: Supports translation between English and German, Spanish, French, Italian, Korean, Dutch, Portuguese, Russian, and Chinese in both directions.
- Context-Aware Translation: Utilizes two preceding source-language context chunks (
Context1andContext2) to improve consistency in document-level translation. - Fixed-Range Chunking: Trained on fixed-range chunks of 256–512 tokens, derived from the sardinelab/DocBlocks dataset.
- ChatML Prompt Format: Fine-tuned with a specific ChatML-style prompt format for translation tasks, ensuring optimal performance with the recommended structure.
Training Details
The model underwent full-parameter supervised fine-tuning for 2 epochs with a learning rate of 7e-6, using a cosine scheduler and AdamW optimizer. It supports a maximum sequence length of 32,768 tokens and was trained in bfloat16 precision. For detailed inference scripts and document chunking procedures, refer to the Doc2FRC GitHub repository.