IlyaGusev/saiga_llama3_8b

Overview

IlyaGusev/saiga_llama3_8b is an 8 billion parameter instruction-tuned model developed by Ilya Gusev, built upon Meta's Llama-3 8B Instruct. It is specifically designed as a Russian-language automatic assistant, capable of engaging in conversations and providing helpful responses in Russian. The model has undergone several iterations of fine-tuning, utilizing various Russian datasets for Supervised Fine-Tuning (SFT) and Preference Optimization (KTO).

Key Capabilities

• Russian Language Proficiency: Optimized for generating high-quality, natural-sounding Russian text.
• Instruction Following: Capable of understanding and executing user instructions in Russian.
• Chatbot Functionality: Designed to act as a conversational assistant, as demonstrated by its system prompt: "Ты — Сайга, русскоязычный автоматический ассистент. Ты разговариваешь с людьми и помогаешь им."
• Iterative Improvement: The model has evolved through multiple versions (v2 to v7), incorporating different SFT and KTO datasets like saiga_scored and lmsys_clean_ru_preferences to enhance performance.
• Llama-3 Prompt Format: Utilizes the native Llama-3 prompt format for optimal interaction, a change from earlier versions that used ChatML.

Good for

• Russian-speaking Chatbots: Ideal for creating conversational AI agents that interact primarily in Russian.
• Content Generation in Russian: Suitable for generating narratives, answering questions, and producing various forms of text content in Russian.
• Research and Development: Provides a strong base for further fine-tuning or experimentation with Russian NLP tasks, especially given its Llama-3 foundation.