Momin-Aldahdouh/MominoMoE-v4
MominoMoE-v4 is a 596 million parameter language model developed by Momin-Aldahdouh, built upon the Qwen3-0.6B base architecture. This model underwent a full fine-tuning process, updating all parameters, with a strong emphasis on coding-first tasks. It is optimized for performance in scenarios requiring efficient processing within its parameter class, making it suitable for specialized applications where a smaller, focused model is beneficial.
Loading preview...
MominoMoE-v4 Overview
MominoMoE-v4 is a 596 million parameter language model developed by Momin-Aldahdouh, representing a fully fine-tuned iteration of the MominoMoE-v3 merged weights, which are based on the Qwen3-0.6B architecture. This version involved updating all 596 million parameters during its training process.
Training Details
The model was trained on a dataset comprising 80,000 training examples and 8,000 validation examples, categorized across 15 distinct categories with a primary focus on coding-related tasks. The training regimen included:
- Steps: 10,000
- Epochs: 4
- Learning Rate: 2e-5 with a cosine schedule
- Precision: bf16
- Final Training Loss: 0.1523
Key Characteristics
MominoMoE-v4 is distinguished by its comprehensive fine-tuning across all parameters, aiming to enhance its capabilities, particularly in coding-centric applications. Its relatively compact size (596M parameters) makes it a candidate for deployment in environments where computational resources are a consideration, while its specialized training suggests proficiency in its targeted domains.
Usage
For users looking to integrate this model, it is available via Ollama, with a specific quantized version (Q4_K_M) provided for ease of use:
ollama run hf.co/Momin-Aldahdouh/MominoMoE-v4:Q4_K_M