raicrits/Hermes7b_ITA
raicrits/Hermes7b_ITA is a 7 billion parameter LLaMa2-based language model, fine-tuned by Stefano Scotta from NousResearch/Nous-Hermes-llama-2-7b. This model is specifically optimized for instruction following in Italian, leveraging a 120k record dataset from raicrits/Orca_ITA_200k. It excels at responding to Italian instructions and can be further fine-tuned for specialized tasks.
Loading preview...
raicrits/Hermes7b_ITA: Italian Instruction-Following LLM
This model is a 7 billion parameter Large Language Model (LLM) developed by Stefano Scotta, specifically fine-tuned to understand and respond to instructions in Italian. It is based on the LLaMa2 architecture, with its foundation being the NousResearch/Nous-Hermes-llama-2-7b model, which itself is an instruction-tuned version of meta-llama/Llama-2-7b.
Key Capabilities
- Italian Instruction Following: The primary strength of Hermes7b_ITA is its ability to process and generate responses to instructions provided in Italian.
- Fine-tuning Potential: While capable of responding to simple instructions out-of-the-box, the model is also suitable for further fine-tuning to adapt it to more specific Italian-language tasks.
- LoRA Fine-tuning: The model was fine-tuned using the efficient LoRA (Low-Rank Adaptation) approach, merging the LoRA adapters with the base model weights.
Training Details
The fine-tuning process involved using 120,000 random instruction/answer pairs from the raicrits/Orca_ITA_200k dataset. Training was conducted over 3 epochs with a learning rate of 2e-4, utilizing mixed-precision training (float16) and a LoRA configuration targeting q_proj and v_proj modules.
Limitations
As with other LLMs, Hermes7b_ITA may generate content that is not factual, or produce biased, offensive, or inappropriate responses.