Sujalvc/akshar-qwen2.5-1.5b-instruct

Akshar Qwen2.5 1.5B Instruct: Romanized Hinglish Specialist

Akshar Qwen2.5 1.5B Instruct is a 1.5 billion parameter instruction-following model, fine-tuned from Qwen/Qwen2.5-1.5B-Instruct, specifically designed for Romanized Hinglish (Hindi-English code-mixed language). It addresses the limitations of general-purpose tokenizers for Indic languages by incorporating a custom tokenizer and unique alignment techniques.

Key Capabilities & Innovations

• Custom Code-Mixed Tokenizer: Utilizes Sujalvc/akshar-32k to significantly reduce sequence lengths for Romanized Indic text, offering up to 1.4x sequence compression and lower inference costs compared to stock Qwen2.5.
• FOCUS Alignment: Employs subword embedding alignment via BPE deconstruction averaging to initialize custom token embeddings, ensuring stability and effective representation learning for new vocabulary merges.
• 0% Devanagari Script Leakage: Enforces normalization rules to strip non-ASCII characters, guaranteeing strictly Romanized Hinglish output.
• Efficient PEFT: Fine-tuned using QLoRA with target adapters on all projection matrices and trainable vocabulary matrices (embed_tokens, lm_head) to integrate new merges effectively.
• High Fertility for Hinglish: Achieves a significantly lower token-per-word fertility rate (1.20-1.50) for Romanized Hinglish compared to general-purpose tokenizers like Llama-3 or GPT-4.

Ideal Use Cases

• Conversational AI in Romanized Hinglish: The first dedicated 1.5B instruction-tuned generative LLM for chat in this specific language profile.
• Applications requiring efficient Hinglish processing: Benefits from reduced token counts, leading to faster generation and lower VRAM usage.
• Educational tools or customer support for Indian users: Can provide explanations, translations, and logical reasoning in a natural, code-mixed style, as demonstrated by qualitative samples in technical concepts, professional context adaptation, and math reasoning.