Overview

arshadshk/Mistral-Hinglish-7B-Instruct-v0.2 is a 7 billion parameter instruction-tuned language model, building upon the Mistral architecture. It has been specifically fine-tuned to excel in Hinglish (Hindi + English) conversational contexts, leveraging a Lora adapter and SFT TRL for training.

Key Capabilities

• Hinglish Conversation: Optimized for generating and understanding responses in Hinglish, a blend of Hindi and English.
• Instruction Following: Designed to follow instructions effectively within a chat-based format, utilizing [INST] and [/INST] tokens.
• Improved Domain Performance: Achieves a test set loss of 0.8, significantly outperforming the base Mistral-Instruct model's loss of 1.8 on the same Hinglish dataset.

Training Details

The model was fine-tuned on 30,000 chat sessions, each with a maximum of 1024 tokens, sourced from the sarvamai/samvaad-hi-v1 dataset. A dedicated test set of 2,000 sessions was used to evaluate its performance.

Instruction Format

Users should format prompts using the [INST] and [/INST] tokens, with the first instruction preceded by a begin-of-sentence ID. This format is supported via the apply_chat_template() method in the Hugging Face Transformers library.

Good For

• Applications requiring natural language understanding and generation in Hinglish.
• Chatbots or conversational AI systems targeting users who communicate in a mix of Hindi and English.
• Research and development in multilingual or code-mixed language models, particularly for Indian languages.