ynklab/Tower-7B-d2d
ynklab/Tower-7B-d2d is a 7 billion parameter, full-parameter fine-tuned version of Unbabel/TowerInstruct-Mistral-7B-v0.2, developed by ynklab. This model is specifically optimized for multilingual document-level machine translation, supporting English to and from German, Spanish, French, Italian, Korean, Dutch, Portuguese, Russian, and Chinese. Trained on the DocBlocks dataset, it excels at direct document-to-document translation with a maximum sequence length of 32,768 tokens.
Loading preview...
Overview
ynklab/Tower-7B-d2d is a 7 billion parameter language model developed by ynklab, specifically fine-tuned for multilingual document-level machine translation. It is based on the Unbabel/TowerInstruct-Mistral-7B-v0.2 architecture and was trained using full-parameter supervised fine-tuning on the sardinelab/DocBlocks dataset.
Key Capabilities
- Document-to-Document Translation: Optimized for translating entire documents rather than isolated sentences.
- Multilingual Support: Translates between English and nine other languages (German, Spanish, French, Italian, Korean, Dutch, Portuguese, Russian, Chinese) in both directions.
- High Context Length: Supports a maximum sequence length of 32,768 tokens, allowing for comprehensive document processing.
- ChatML Prompting: Utilizes a specific ChatML-style prompt format for translation tasks, as detailed in the Doc2FRC GitHub repository.
Training Details
The model was fine-tuned for 2 epochs with a learning rate of 7e-6, using bfloat16 precision and an AdamW optimizer. Its training focused on the DocBlocks dataset, which is designed for document-level parallel data.
Good For
- Developers and researchers requiring robust document-level translation capabilities.
- Applications needing accurate translation between English and the supported languages, especially for longer texts.
- Integration into systems where the specific prompt format and high context length are beneficial for translation tasks.