tartuNLP/Llamma-2-7b-ukr-p1-llama-errors-p2-GEC
The tartuNLP/Llamma-2-7b-ukr-p1-llama-errors-p2-GEC model is a 7 billion parameter Llama-2-based language model developed by TartuNLP, specifically fine-tuned for Grammatical Error Correction (GEC) in Ukrainian. It was trained on a unique dataset comprising 1 million synthetic errors generated by a Llama-based error generation model, combined with human GEC data. This model excels at identifying and correcting grammatical and spelling errors in Ukrainian text, offering a specialized solution for high-quality text refinement. Its 4096-token context length supports processing moderately sized Ukrainian sentences for error correction.
Loading preview...
Model Overview
The tartuNLP/Llamma-2-7b-ukr-p1-llama-errors-p2-GEC is a 7 billion parameter Llama-2-based model developed by TartuNLP, specifically engineered for Grammatical Error Correction (GEC) in Ukrainian. This model stands out due to its unique training methodology, which involved fine-tuning on a substantial dataset of 1 million synthetic errors generated by a Llama-based error generation model, complemented by human-annotated GEC data.
Key Capabilities
- Ukrainian GEC: Specializes in identifying and correcting grammatical and spelling errors in Ukrainian text.
- Llama-2 Architecture: Built upon the Llama-2-7b-Ukrainian base, leveraging its robust language understanding capabilities.
- Synthetic Error Training: Utilizes a novel approach of training on automatically generated errors, which have been shown to mimic human errors effectively.
- Performance: Outperforms previous state-of-the-art error correction models, achieving significant gains (0.8 to 6 F0.5 points) across various languages, including Ukrainian, as detailed in the associated research paper.
Good For
- Automated Ukrainian Text Correction: Ideal for applications requiring high-quality grammatical and spelling correction for Ukrainian content.
- Research in GEC: Provides a strong baseline and a valuable resource for further research into error generation and correction techniques, particularly for less-resourced languages.
- Developers: Offers a ready-to-use pipeline for integrating Ukrainian GEC functionality into applications, with clear inference examples provided.
For more in-depth information on the training and inference code, refer to the TartuNLP GEC-LLM repository. The underlying research is detailed in the paper "To Err Is Human, but Llamas Can Learn It Too" presented at EMNLP 2024, available on arXiv.