laion/r2egym-31600__Qwen3-8B
The laion/r2egym-31600__Qwen3-8B model is an 8 billion parameter language model, fine-tuned from the Qwen/Qwen3-8B architecture. It was trained on the /e/data1/datasets/playground/ot/hf_hub/datasets--laion--r2egym-unified-31600 dataset, suggesting a specialization in areas related to the 'r2egym-unified' data. With a context length of 32768 tokens, this model is designed for tasks requiring extensive contextual understanding.
Loading preview...
Model Overview
This model, laion/r2egym-31600__Qwen3-8B, is an 8 billion parameter language model derived from the Qwen/Qwen3-8B base architecture. It has been specifically fine-tuned on the /e/data1/datasets/playground/ot/hf_hub/datasets--laion--r2egym-unified-31600 dataset.
Training Details
The fine-tuning process involved several key hyperparameters:
- Learning Rate: 4e-05
- Batch Size: A
train_batch_sizeof 1 andeval_batch_sizeof 8 were used, with atotal_train_batch_sizeof 96 across 32 devices. - Optimizer: AdamW_Torch_Fused with betas=(0.9, 0.98) and epsilon=1e-08.
- Scheduler: A cosine learning rate scheduler with a warmup ratio of 0.1.
- Epochs: The model was trained for 7.0 epochs.
Framework Versions
The training environment utilized:
- Transformers 4.57.6
- Pytorch 2.9.1+cu130
- Datasets 4.7.0
- Tokenizers 0.22.2
Potential Use Cases
Given its fine-tuning on a specific dataset, this model is likely optimized for tasks related to the content and structure of the r2egym-unified-31600 dataset. Developers should evaluate its performance on tasks that align with this dataset's characteristics.