OpenThinker3-1.5B Overview

OpenThinker3-1.5B is a 1.5 billion parameter language model, fine-tuned by cjiao from the base Qwen/Qwen2.5-1.5B-Instruct architecture. This model has been specifically trained on the open-thoughts/OpenThoughts-114k dataset, indicating a specialization in tasks related to open-ended ideation, discussion, or complex reasoning.

Key Training Details

The model underwent fine-tuning with the following hyperparameters:

• Learning Rate: 0.00016
• Batch Size: A total training batch size of 256 (16 per device with 8 gradient accumulation steps across 2 GPUs).
• Optimizer: AdamW with standard betas and epsilon.
• Scheduler: Cosine learning rate scheduler with a 0.1 warmup ratio.
• Epochs: Trained for 3 epochs.

Potential Use Cases

Given its fine-tuning on a dataset focused on 'open thoughts', OpenThinker3-1.5B is likely well-suited for:

• Generating creative text or brainstorming ideas.
• Assisting in tasks requiring nuanced understanding of discussions.
• Applications where the model needs to explore various perspectives or solutions.