leonardPKU/qwen25vl_3b_full_1050k_0429_1ep_bsz64_lr1e-5_train_full
The leonardPKU/qwen25vl_3b_full_1050k_0429_1ep_bsz64_lr1e-5_train_full is a 3 billion parameter model trained from scratch. This model's specific architecture and primary differentiators are not detailed in the provided information, indicating it may be a base model or an experimental variant. Its key characteristics and primary use case are not explicitly defined, suggesting further evaluation is needed to determine its optimal applications.
Loading preview...
Model Overview
This model, qwen25vl_3b_full_1050k_0429_1ep_bsz64_lr1e-5_train_full, is a 3 billion parameter model that was trained from scratch. The specific dataset used for training is not detailed, and further information regarding its architecture, intended uses, and limitations is currently needed.
Training Details
The model underwent training with the following key hyperparameters:
- Learning Rate: 1e-05
- Total Training Batch Size: 64 (achieved with
train_batch_sizeof 2 andgradient_accumulation_stepsof 4 across 8 devices) - Optimizer: AdamW with default betas and epsilon
- LR Scheduler: Cosine type with a warmup ratio of 0.05
- Training Steps: 20,000
Framework Versions
The training utilized:
- Transformers 4.51.2
- Pytorch 2.6.0+cu124
- Datasets 3.4.1
- Tokenizers 0.21.0
Good for
- Experimental purposes: Given the lack of specific use case information, this model is suitable for researchers and developers looking to experiment with a 3B parameter model trained from scratch.
- Further fine-tuning: It can serve as a base model for domain-specific fine-tuning once its base capabilities are better understood.