Hugging Face Overview
Meno-Tiny-0.1: A Russian-Optimized Language Model
Meno-Tiny-0.1 is a 1.5 billion parameter language model, a fine-tuned descendant of Qwen2.5-1.5B-Instruct, developed by Ivan Bondarenko. Its name, "Meno," reflects its specialization in question answering from text, particularly within Retrieval-Augmented Generation (RAG) pipelines. The model leverages a Transformer architecture with features like SwiGLU activation and group query attention.
Key Capabilities
- Russian Language Proficiency: Specifically "Russified" through fine-tuning on a dedicated Russian instruct dataset, while retaining English language capabilities.
- Diverse NLP Tasks: Excels at a range of tasks including:
- Answering questions about text
- Text summarization
- Determining and detoxifying text toxicity
- Anaphora resolution in dialogues
- Correcting speech recognition errors
- Performance on Russian Benchmarks: Achieves a 0.365 overall score on the MERA benchmark, an independent evaluation for Russian language models, outperforming its base model. It also scores 0.399 / 0.29 on the MultiQ task, crucial for RAG effectiveness.
Good For
- Memory/Compute-Constrained Environments: Designed for efficient operation in resource-limited settings.
- Latency-Bound Scenarios: Suitable for applications requiring quick response times.
- Retrieval Augmented Generation (RAG): Serves as a strong building block for RAG pipelines, particularly for Russian-language content.
- Research and Commercial Use: Intended for both research and commercial applications requiring robust Russian NLP capabilities.