ahmedheakl/cass-smA100-7b
ahmedheakl/cass-smA100-7b is a 7.6 billion parameter language model fine-tuned from Qwen/Qwen2.5-Coder-7B-Instruct. This model is part of the CASS collection, focusing on specific optimizations for its base architecture. It is intended for tasks aligned with its Qwen2.5-Coder foundation, though specific differentiators are not detailed.
Loading preview...
Model Overview
ahmedheakl/cass-smA100-7b is a 7.6 billion parameter language model, fine-tuned from the Qwen/Qwen2.5-Coder-7B-Instruct base model. This model is part of the broader CASS collection, suggesting a specialized focus or application within that framework. While specific details about the fine-tuning dataset and its unique capabilities are not provided, its origin from a 'Coder' model implies a potential aptitude for code-related tasks.
Training Details
The model underwent a fine-tuning process using the following hyperparameters:
- Learning Rate: 2e-05
- Batch Size: 2 (train), 8 (eval)
- Gradient Accumulation: 8 steps, leading to a total train batch size of 112
- Optimizer: ADAMW_TORCH
- Scheduler: Cosine with 0.1 warmup ratio
- Epochs: 3.0