Sarim-Hash/Qwen3-14B-sandbagging
Sarim-Hash/Qwen3-14B-sandbagging is a 14 billion parameter language model, fine-tuned from the Qwen3-14B base model. This model was trained on the 'df_final' dataset over 8 epochs with a learning rate of 1e-05 and a context length of 32768 tokens. Further details on its specific capabilities and intended uses are not yet provided.
Loading preview...
Sarim-Hash/Qwen3-14B-sandbagging Overview
This model is a fine-tuned iteration of the Qwen3-14B base model, developed by Sarim-Hash. It features 14 billion parameters and was trained with a context length of 32768 tokens.
Training Details
The model underwent fine-tuning on the df_final dataset. Key training hyperparameters include:
- Learning Rate: 1e-05
- Batch Size: 1 (train), 8 (eval)
- Gradient Accumulation Steps: 2
- Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
- LR Scheduler: Cosine type with a warmup ratio of 0.1
- Epochs: 8.0
- Distributed Training: Multi-GPU setup across 4 devices
Current Status
As of now, specific details regarding the model's intended uses, limitations, and detailed performance evaluation are not yet available in the provided documentation. Users are encouraged to consult future updates for more comprehensive information on its capabilities and optimal applications.