SemanticAlignment/Llama-3.1-8B-Italian-SAVA
Llama-3.1-8B-Italian-SAVA is an 8 billion parameter Llama-3.1-8B-Adapted model developed by SapienzaNLP, ISTI-CNR, and ILC-CNR. This continually trained auto-regressive language model features a substituted tokenizer, aligning with Minerva-3B, and is specifically adapted for Italian language tasks. It was trained on a skewed dataset from CulturaX, prioritizing Italian data, making it suitable for Italian text generation and understanding.
Loading preview...
Model Overview
Llama-3.1-8B-Italian-SAVA is an 8 billion parameter large language model from the Llama-3.1-8B-Adapted collection, developed by SapienzaNLP, ISTI-CNR, and ILC-CNR. It is an auto-regressive model built on an optimized transformer architecture. A key characteristic of this model is its continual training after tokenizer substitution, where its tokenizer is aligned with that of Minerva-3B.
Key Capabilities
- Italian Language Adaptation: Specifically adapted for Italian, making it proficient in generating and understanding Italian text.
- Tokenizer Alignment: Utilizes a tokenizer similar to Minerva-3B, potentially offering efficiencies for Italian language processing.
- Optimized for Italian Data: Trained on a curated dataset from CulturaX, with a significant skew towards Italian content (a 1:4 English to Italian ratio).
Good For
- Italian Text Generation: Ideal for applications requiring high-quality text generation in Italian.
- Italian NLP Tasks: Suitable for various natural language processing tasks where Italian language proficiency is crucial.
- Research on Vocabulary Adaptation: Relevant for researchers interested in the effects of tokenizer substitution and vocabulary adaptation for specific languages, as detailed in the associated paper "Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptation".