lucasjoy88/Gemma-SEA-LION-v3-9B-IT-Cebuano
The lucasjoy88/Gemma-SEA-LION-v3-9B-IT-Cebuano is a 9.3 billion parameter Gemma 2-based causal language model, fine-tuned by Lucas Joyce to specialize in Cebuano-English lexical lookup. This model integrates a Cebuano dictionary LoRA at full strength, significantly improving its vocabulary for the Cebuano language, which is poorly served by general-purpose LLMs. It excels at providing accurate Cebuano-English glosses, particularly for rare words, by installing verified lexical knowledge from attestation-based dictionaries.
Loading preview...
Overview
The lucasjoy88/Gemma-SEA-LION-v3-9B-IT-Cebuano is a specialized 9.3 billion parameter Gemma 2-based causal language model developed by Lucas Joyce. It enhances the aisingapore/Gemma-SEA-LION-v3-9B-IT base model by merging a Cebuano (Binisaya) dictionary LoRA at full strength. This model was trained on a single desktop machine without cloud resources, focusing on improving Cebuano-English lexical lookup.
Key Capabilities
- Cebuano-English Lexical Lookup: Significantly improves the base model's ability to provide correct glosses for Cebuano words, including rare headwords, by integrating vocabulary from two attestation-based dictionaries.
- Low-Resource Language Support: Addresses the poor performance of general-purpose LLMs for Cebuano, a language spoken by approximately 20 million people.
- Direct Dictionary Query: Optimized for specific query phrasings like "What does 'X' mean in Cebuano?" or "Unsa ang kahulogan sa pulong nga 'X'?"
Limitations and Considerations
- Conversational Ability: Fine-tuning for dictionary lookup partly overwrote the base model's conversational skills, leading to looping or drifting into Tagalog in chat interfaces.
- Off-Template Query Sensitivity: Retrieval accuracy is highly dependent on query phrasing; variations can lead to incorrect glosses, often from other genuine dictionary entries.
- Lexicographer Prose: Glosses are presented in lexicographer prose, which may not be ideal for language learners.
- Refusal Reflex: The fine-tuning process degraded the base model's ability to refuse non-words, sometimes leading to invented definitions.
When to Use This Model
This model is ideal for applications requiring precise Cebuano-English lexical lookup, especially for dictionary-style queries. Users should adhere to the trained phrasing and consider answers for rare words as leads for verification due to the model's sensitivity to query variations and its tendency to invent for non-words.