Luma-base by Frostie08 is a 4-billion parameter causal language model, based on the Qwen3-4B architecture, specifically pre-trained for Haitian Creole (Kreyòl Ayisyen). It features a 32768-token context length and is optimized for tasks like speech-to-text correction, translation, and text generation in Haitian Creole. The model achieved a final validation loss of 1.9252 and a perplexity of approximately 6.8, demonstrating high confidence in Haitian Creole word prediction.
Loading preview...
Luma-base: Haitian Creole Foundation Model
Luma-base, developed by Frostie08, is a 4-billion parameter causal language model built upon the Qwen3-4B architecture. Its primary distinction lies in its specialized and extensive pre-training on a high-quality Haitian Creole corpus, kani-pretrain, making it a dedicated resource for the language.
Key Capabilities & Features
- Haitian Creole Specialization: Deeply understands the nuances, grammar, and cultural context of Haitian Creole (ht-HT).
- Efficient Training: Utilized the Unsloth library for training, ensuring efficiency and mathematical precision.
- Performance Metrics: Achieved a final validation loss of 1.9252 and a perplexity of ~6.8, indicating strong language modeling capabilities for Haitian Creole.
- Robust Architecture: Based on the Qwen3-4B model, providing a solid foundation for language tasks.
Ideal Use Cases
Luma-base is designed to serve as a core engine for various applications requiring high-quality Haitian Creole processing:
- Speech-to-Text (STT) Correction: Enhancing accuracy in transcribing spoken Haitian Creole.
- Machine Translation: Improving translation quality between Haitian Creole and other languages.
- Text Generation: Creating coherent and contextually relevant text in Haitian Creole.
- Language Research: Providing a robust base model for further research and development in Haitian Creole NLP.