jahyungu/Qwen2.5-1.5B-Instruct_mbpp
jahyungu/Qwen2.5-1.5B-Instruct_mbpp is a 1.5 billion parameter instruction-tuned causal language model, fine-tuned from Qwen/Qwen2.5-1.5B-Instruct. This model has a context length of 32768 tokens and was trained with specific hyperparameters including a learning rate of 1e-05 and 5 epochs. Its primary differentiator and specific use cases are not detailed in the provided information, indicating a general-purpose instruction-following model.
Loading preview...
Model Overview
This model, jahyungu/Qwen2.5-1.5B-Instruct_mbpp, is a 1.5 billion parameter instruction-tuned language model. It is a fine-tuned variant of the base model Qwen/Qwen2.5-1.5B-Instruct.
Key Characteristics
- Base Model: Fine-tuned from Qwen/Qwen2.5-1.5B-Instruct.
- Parameter Count: 1.5 billion parameters.
- Context Length: Supports a context length of 32768 tokens.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 1e-05
- Batch Size: A train batch size of 2 and an eval batch size of 8, with a total effective batch size of 16 due to gradient accumulation steps of 8.
- Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08.
- Scheduler: Cosine learning rate scheduler with a warmup ratio of 0.03.
- Epochs: Trained for 5 epochs.
- Frameworks: Utilized Transformers 4.55.0, PyTorch 2.6.0+cu124, Datasets 3.4.1, and Tokenizers 0.21.0.
Intended Use Cases
Specific intended uses and limitations are not detailed in the provided information. As an instruction-tuned model, it is generally suitable for a variety of natural language understanding and generation tasks where instruction following is key, though its specific strengths are not explicitly stated.