sainived656/soreqen-s1-mega
SoreQen S1 Mega is a 4.5 billion parameter bilingual (English/Hinglish) assistant developed by ZorQelis AI, fine-tuned from Qwen/Qwen3.5-4B. This model specializes in conversational AI, offering improved identity retention without system prompts, enhanced reasoning, and better tool-calling capabilities compared to its base model. It is optimized for engaging in natural language conversations in both English and Romanized Hinglish, making it suitable for applications requiring multilingual chat support.
Loading preview...
SoreQen S1 Mega: Bilingual Conversational Assistant
SoreQen S1 Mega, developed by ZorQelis AI, is a 4.5 billion parameter language model fine-tuned from Qwen/Qwen3.5-4B. It is designed as a bilingual assistant, proficient in both English and Romanized Hinglish, with a focus on conversational interactions.
Key Capabilities & Enhancements
This model was fine-tuned using a single low-rank adaptation (LoRA, r=16) on 25,080 supervised examples, primarily targeting assistant identity and English/Hinglish conversation. While the vision encoder, multimodal projector, and embedding tables remain frozen from the base model, SoreQen S1 Mega demonstrates several key improvements:
- Improved Identity: Significantly better at maintaining its identity without an explicit system prompt (6/6 vs. 0/6 for the base model).
- Enhanced Reasoning: Shows better performance in reasoning tasks (5/5 vs. 4/5 for the base model).
- Better Tool Calling: Exhibits improved capabilities in tool calling (2/2 vs. 1/2 for the base model).
- Bilingual Proficiency: Excels in conversational Hinglish (Roman script) and English, adapting to the user's language and register.
- Context Window: Inherits a substantial context window of 262,144 tokens from its base model.
- Vision Tower: Retains the vision tower from the base model, though it was frozen during fine-tuning.
Intended Use Cases
SoreQen S1 Mega is particularly well-suited for:
- Bilingual Chatbots: Applications requiring conversational AI in both English and Romanized Hinglish.
- Customer Support: Providing assistance where users might switch between English and Hinglish.
- Interactive Assistants: Scenarios benefiting from a model with strong identity retention and reasoning in conversational contexts.
Limitations
It's important to note that as a smaller model, it may confidently state unverified numbers. Hinglish output is exclusively in Roman script, not Devanagari. The model is trained for conversation and not intended for safety-critical or professional advice.