ameer4wisam/gemma-iraqi-finetune-v2
The ameer4wisam/gemma-iraqi-finetune-v2 is a Gemma-4-E12B-it based language model, fine-tuned using LoRA for Iraqi Arabic dialogue. This model specializes in short, direct responses for sales, services, and daily life conversations, mimicking an Iraqi seller's style. It achieves a validation loss of 0.215 and a token accuracy of 93.8% after one epoch of training on 163,429 conversations. The model is optimized for specific inference settings to ensure coherent and factual outputs, particularly for numerical data.
Loading preview...
Overview
This model, ameer4wisam/gemma-iraqi-finetune-v2, is a fine-tuned version of google/gemma-4-E12B-it specifically designed for Iraqi Arabic conversations. It was trained using LoRA (r=16, alpha=32, dropout=0.05) and then merged into the base model's bf16 weights. The primary focus is on sales, services, and daily life interactions, with an emphasis on short, direct responses characteristic of an Iraqi seller.
Key Capabilities & Features
- Iraqi Dialect Specialization: Proficient in common Iraqi phrases and conversational style.
- Optimized for Specific Domains: Excels in dialogues related to buying, selling, and service inquiries.
- Robust Inference Settings: Requires specific inference parameters (
attn_implementation="eager",max_new_tokens=64,repetition_penaltydisabled) to prevent incoherent outputs. - Factual Consistency: Designed to handle numerical data and facts reliably when used with a dynamic catalog and deterministic inference (
do_sample=False). - Context Retention: Demonstrates good context understanding over multiple turns, especially with a system prompt.
Limitations & Considerations
- Numerical Accuracy: Unreliable for numbers without an injected catalog; calculations like discounts must be pre-computed.
- Randomness Impact: High
temperaturevalues can degrade factual accuracy; deterministic generation is crucial for factual responses. - Generalization: Performance outside of sales/service contexts may vary.
- System Prompt Dependency: While it responds well to system prompts, its core training did not include them, relying on the base Gemma's capabilities.