Overview

Zefiro-7b-beta-ITA-v0.1 is a 7 billion parameter GPT-like model developed by giux78 and funded by Business Operating System. It is a supervised fine-tuned (SFT) model specifically designed for the Italian language, built upon the mistralai/Mistral-7B-v0.1 base model. The project's aim is to contribute to open-source models and datasets tailored for Italian.

Key Capabilities

• Italian Language Generation: Primarily focused on generating text in Italian.
• Conversational Tasks: Can be used as a base model for more specific conversational applications in Italian.
• Fine-tuned Approach: Utilizes the Zephyr fine-tuning methodology, adapted for Italian, and incorporates insights from the LLaMAntino model.

Training Details

The model was fine-tuned on a filtered and preprocessed version of the UltraChat-ITA dataset, which is derived from the UltraChat dataset containing synthetic dialogues generated by ChatGPT. The developers acknowledge that the translation phase for creating Italian datasets is critical and can introduce inaccuracies.

Limitations

• Safety Alignment: The model has not undergone RLHF (Reinforcement Learning from Human Feedback) for safety alignment, meaning it may produce problematic outputs.
• Base Model Corpus: The exact size and composition of the training corpus for the base mistralai/Mistral-7B-v0.1 model are unknown, but likely include web data and technical sources.