Overview

This model, mlfoundations-dev/openr1_codeforces, is a specialized fine-tune of the Qwen/Qwen2.5-7B-Instruct base model. It has been adapted using the mlfoundations-dev/openr1_codeforces dataset, suggesting an optimization for tasks relevant to that specific data domain.

Training Details

The fine-tuning process involved several key hyperparameters:

• Learning Rate: 4e-05
• Batch Size: 1 (train), 8 (eval)
• Gradient Accumulation Steps: 4, leading to a total effective batch size of 128
• Optimizer: ADAMW_TORCH with standard betas and epsilon
• LR Scheduler: cosine with a 0.1 warmup ratio
• Epochs: 5.0

Intended Uses

Given its fine-tuning on the mlfoundations-dev/openr1_codeforces dataset, this model is likely intended for applications within the scope of that dataset's content. Users should refer to the dataset's documentation for specific use cases and potential limitations.