MoxoffSrL/Azzurro
Azzurro is a 7 billion parameter causal language model developed by MoxoffSrL, fine-tuned from Mistral-7B-v0.2. It is specifically designed for contextual understanding and Retrieval Augmented Generation (RAG) tasks, leveraging SFT and LoRA adjustments. The model is trained on public and in-house datasets, excelling in applications requiring strong contextual awareness. It achieves an average score of 0.52 on the Open Ita LLM Leaderboard evaluation metrics.
Loading preview...
Overview
Azzurro is a 7 billion parameter language model developed by MoxoffSrL, built upon the Mistral-7B-v0.2 architecture. It has undergone specific fine-tuning using Supervised Fine-Tuning (SFT) and LoRA adjustments, leveraging both publicly available datasets like SQUAD-it and proprietary in-house datasets. This model is particularly optimized for understanding and maintaining context, making it well-suited for advanced natural language processing tasks.
Key Capabilities
- Contextual Understanding: Designed to grasp and retain context effectively, crucial for coherent and relevant responses.
- Retrieval Augmented Generation (RAG): Ideal for applications where generating responses requires integrating information from external knowledge bases.
- Italian Language Proficiency: Evaluated against benchmarks used for the Open Ita LLM Leaderboard, demonstrating performance in Italian language tasks.
Performance
On the Open Ita LLM Leaderboard evaluation, Azzurro achieved the following scores:
hellaswag_it acc_norm: 0.6067arc_it acc_norm: 0.4405m_mmlu_it 5-shot acc: 0.5112- Average Score: 0.52
Limitations
As Azzurro has not undergone alignment for human preferences or safety through RLHF, it may produce problematic outputs, especially when explicitly prompted to do so. The exact composition of the base Mistral-7B-v0.2 training corpus is unknown but likely includes a mix of web data and technical sources.
Good For
- Developing applications that require strong contextual awareness.
- Implementing RAG systems for information retrieval and generation.
- Tasks involving the Italian language, given its evaluation on relevant benchmarks.