Avi 1.1B: A Hebrew-Speaking TinyLlama Fine-tune

ashercn97/avi-1.1b is a specialized language model developed by ashercn97, built upon the compact and efficient TinyLlama architecture. While the base TinyLlama model is English-centric, Avi 1.1B has undergone a targeted fine-tuning process to acquire proficiency in the Hebrew language.

Key Capabilities

• Hebrew Language Processing: Avi 1.1B is capable of understanding and generating text in Hebrew, a significant departure from its English-only predecessor.
• Fine-tuned from TinyLlama: Leverages the efficient architecture of TinyLlama, making it suitable for applications where a smaller model footprint is advantageous.

Training Details

• The model was trained using Hebrew translations of the Platypus and Alpaca-GPT4 datasets. This approach, while acknowledged by the developer as not optimal, was the method employed for its Hebrew language acquisition.

Good For

• Applications requiring a compact language model with Hebrew language capabilities.
• Experimentation with multilingual fine-tuning techniques on smaller models.

Limitations

• The training methodology involved machine translation of datasets, which may introduce artifacts or limitations compared to models trained on native Hebrew corpora.