GenVRadmin/AryaBhatta-GemmaOrca-Merged

Model Overview

GenVRadmin's AryaBhatta-GemmaOrca-Merged is an 8.5 billion parameter language model built upon Google's Gemma architecture. This model is part of the AryaBhatta series, specifically designed to enhance reasoning and mathematical capabilities, and to support multiple Indian languages alongside English.

Key Capabilities & Training

• Enhanced Reasoning & Math: The model was initially fine-tuned using Microsoft's Orca datasets, including a Hindi-specific Orca maths dataset, to significantly improve its mathematical and reasoning scores. This process boosted the MATHS score from 24.3 (Gemma-7B) to 31.6 (GemmaOrca).
• Multilingual Support: It has been extensively fine-tuned on 9 Indian languages (Hindi, Tamil, Punjabi, Bengali, Gujarati, Oriya, Telugu, Kannada, Malayalam) in addition to English, utilizing GenVR's Samvaad datasets and various open-source Indic language datasets.
• Benchmark Performance: AryaBhatta-GemmaOrca demonstrates competitive performance across several benchmarks, achieving an average score of 50.59, outperforming zephyr-7b-gemma-v0.1 and google/gemma-7b-it.

Use Cases

This model is particularly well-suited for applications requiring strong mathematical reasoning and multilingual understanding, especially within the context of Indian languages. Its fine-tuning on diverse datasets makes it a robust choice for tasks involving instruction following and general language generation in these linguistic domains.