IlyaGusev/saiga_nemo_12b

Overview

IlyaGusev/saiga_nemo_12b is a Russian-language fine-tuned model derived from an abliterated version of Mistral Nemo. Developed by IlyaGusev, this model is designed to function as an automatic assistant, engaging in conversations and assisting users with various prompts in Russian. It has undergone several iterations, with v3 being the latest, incorporating Supervised Fine-Tuning (SFT) and SimPO (Simple Preference Optimization) using specific dataset and model configurations.

Key Capabilities

• Russian Language Proficiency: Specialized for high-quality Russian text generation and understanding.
• Conversational AI: Designed to act as a helpful assistant, capable of engaging in dialogues.
• Instruction Following: Responds to user queries and instructions, as demonstrated by examples like explaining why grass is green or generating creative stories.
• Prompt Format Flexibility: Supports different prompt formats (v3, v1, v2) based on the original Mistral Nemo format, with system prompts at the beginning.
• Accessibility: Provided with llama.cpp GGUF versions and a Colab notebook for easy experimentation and deployment.

Performance

The model's performance is evaluated on Russian-specific benchmarks:

• RuArenaHard: Scores are provided for v1, v2, and v3, showing iterative improvements.
• PingPong: Metrics for conversational turn-taking are also presented for each version.

Use Cases

• Russian Chatbots: Ideal for building conversational agents that interact in Russian.
• Content Generation: Can be used for generating diverse Russian text, from factual explanations to creative narratives.
• Language Assistance: Suitable for applications requiring an automated assistant for Russian-speaking users.