Model Overview

The cjiao/OpenThoughts3-greedy-groups-top-openthinker3-1.5B-checkpoint-375 is a 1.5 billion parameter language model. It is a fine-tuned version of the cjiao/OpenThinker3-1.5B-checkpoint-375 base model, specifically adapted using the cjiao/OpenThoughts3-greedy-groups-top-openthinker3-1.5B-checkpoint-375 dataset.

Training Details

The model was trained for 1 epoch with a learning rate of 0.00016, using a total batch size of 256 (achieved with train_batch_size 8 and gradient_accumulation_steps 16) across 2 GPUs. The optimizer used was adamw_torch with standard betas and epsilon, and a cosine learning rate scheduler. The training utilized Transformers 4.46.1, Pytorch 2.5.1+cu121, Datasets 3.1.0, and Tokenizers 0.20.3.

Current Status

As of this release, specific details regarding the model's intended uses, limitations, and detailed evaluation data are not yet provided by the developer. Users should refer to future updates for more comprehensive information on its capabilities and optimal applications.