Krystalan/DRT-7B

TEXT GENERATIONConcurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Dec 23, 2024License:cc-by-nc-sa-4.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

Krystalan/DRT-7B is a 7.6 billion parameter causal language model built upon the Qwen2.5-7B-Instruct backbone, specifically fine-tuned for neural machine translation (MT). It specializes in "Deep Reasoning Translation" by generating a long chain-of-thought before producing the final translation, particularly for complex English sentences containing similes or metaphors. This model aims to explore and facilitate long thought reasoning in MT, demonstrating improved BLEU and CometScore performance compared to its backbone and other 7B-14B models.

Loading preview...

DRT-7B: Deep Reasoning Translation via Long Chain-of-Thought

DRT-7B is a 7.6 billion parameter language model developed by Krystalan, fine-tuned from the Qwen2.5-7B-Instruct backbone. It is part of the DRT (Deep Reasoning Translation) family, which explores integrating long chain-of-thought reasoning into neural machine translation (MT). The model's core innovation lies in its approach to handling complex translation tasks, particularly those involving nuanced language like similes and metaphors.

Key Capabilities & Features

  • Deep Reasoning Translation: DRT-7B generates a detailed reasoning process (thought) before providing the final translation, mimicking human-like problem-solving for complex linguistic structures.
  • Specialized Training Data: The model was trained on 22,264 synthesized MT samples, specifically mined from literature for English sentences containing similes or metaphors suitable for long thought translation.
  • Multi-Agent Synthesis Framework: Training data was generated using a novel multi-agent framework comprising a translator, an advisor, and an evaluator, designed to create high-quality, long-thought MT samples.
  • Improved Translation Metrics: DRT-7B demonstrates enhanced performance in translation quality, achieving a BLEU score of 35.54 and a CometScore of 80.19, outperforming its Qwen2.5-7B-Instruct backbone and other comparable models like Llama-3.1-8B-Instruct and Marco-o1-7B on reported benchmarks.

Use Cases & Differentiation

DRT-7B is primarily designed for research and applications requiring high-quality, nuanced machine translation, especially for texts that benefit from deeper semantic understanding and reasoning. It differentiates itself by explicitly modeling and outputting the reasoning process, making it suitable for tasks where interpretability and accuracy in complex linguistic contexts are paramount. While not aiming to surpass general-purpose large models like OpenAI's O1 in broad MT performance, it offers a unique approach to integrating long thought into MT, facilitating further research in this direction.

Finetunes
1 models