ahmedheakl/gg-armv8-O0
The ahmedheakl/gg-armv8-O0 model is a 1.5 billion parameter instruction-tuned causal language model, fine-tuned by ahmedheakl from the Qwen2.5-Coder-1.5B-Instruct base. This model specializes in code-related tasks, having been trained on the stack_16k, anghabench_16k_1, and anghabench_16k_2 datasets. It demonstrates strong performance in its specialized domain, achieving a final evaluation loss of 0.0003, making it suitable for code generation and understanding applications.
Loading preview...
Model Overview
The ahmedheakl/gg-armv8-O0 is a 1.5 billion parameter instruction-tuned model, developed by ahmedheakl. It is a fine-tuned variant of the Qwen/Qwen2.5-Coder-1.5B-Instruct base model, specifically optimized for code-related tasks.
Key Characteristics
- Base Model: Qwen2.5-Coder-1.5B-Instruct.
- Parameter Count: 1.5 billion parameters.
- Context Length: 32768 tokens.
- Fine-tuning Datasets: Trained on
stack_16k,anghabench_16k_1, andanghabench_16k_2datasets, indicating a focus on code-centric data. - Performance: Achieved a low evaluation loss of 0.0003, suggesting high accuracy on its specialized tasks.
Training Details
The model was trained with a learning rate of 1e-05 over 2 epochs, utilizing a multi-GPU setup with 2 devices and a total batch size of 8. The training process involved 300,000 steps, showing consistent reduction in loss from 0.0037 to 0.0002.
Intended Use Cases
Given its fine-tuning on code-related datasets, this model is primarily intended for applications requiring code generation, completion, or understanding. Its compact size (1.5B parameters) combined with a substantial context length (32768 tokens) makes it efficient for handling moderately complex coding challenges.