occiglot/occiglot-7b-it-en
Occiglot-7B-IT-EN is a 7 billion parameter generative language model developed by the Occiglot Research Collective. Based on Mistral-7B-v0.1, it was continually pre-trained on 113 billion additional multilingual and code tokens, with a focus on Italian and English. This model is a general-purpose base model designed for Italian and English language tasks, not instruction-tuned for chat applications.
Loading preview...
Occiglot-7B-IT-EN: A Polyglot Base Model for Italian and English
Occiglot-7B-IT-EN is a 7 billion parameter generative language model developed by the Occiglot Research Collective. It is built upon the Mistral-7B-v0.1 architecture and has undergone extensive continued pre-training on 113 billion tokens of additional multilingual and code data, with a block size of 8,192 tokens. The training data distribution was approximately 52% Italian, 34% English, and 13% code.
Key Capabilities
- Bilingual Focus: Optimized for performance in both Italian and English.
- Code Understanding: Includes a significant portion of code in its training data, enhancing its capabilities for programming-related tasks.
- Base Model: Serves as a robust general-purpose base model, suitable for further fine-tuning or specific applications.
- Research Project: Represents an ongoing open research initiative for multilingual language models.
Good For
- Developers and researchers working on Italian and English NLP applications.
- As a foundational model for tasks requiring strong bilingual understanding in Italian and English.
- Continued research and development in multilingual language models, particularly for Western European languages.