laion/GLM-4.6-stackexchange-overflow-sandboxes-32eps-65k-reasoning_global-batch-size_32_Qwen3-32B
The laion/GLM-4.6-stackexchange-overflow-sandboxes-32eps-65k-reasoning_global-batch-size_32_Qwen3-32B model is a 32 billion parameter language model fine-tuned from Qwen/Qwen3-32B. It was trained on the open-athena/GLM-4.6-stackexchange-overflow-sandboxes-32eps-65k-reasoning dataset, suggesting a specialization in content related to Stack Exchange and overflow sandboxes. With a context length of 32768 tokens, this model is likely optimized for understanding and generating detailed technical discussions and problem-solving content.
Loading preview...
Model Overview
This model, GLM-4.6-stackexchange-overflow-sandboxes-32eps-65k-reasoning_global-batch-size_32_Qwen3-32B, is a specialized large language model based on the Qwen3-32B architecture. It has been fine-tuned by laion on a specific dataset: open-athena/GLM-4.6-stackexchange-overflow-sandboxes-32eps-65k-reasoning.
Training Details
The fine-tuning process involved several key hyperparameters:
- Learning Rate:
4e-05 - Batch Size:
1(train),8(eval), with a total effective training batch size of32due to gradient accumulation. - Optimizer:
ADAMW_TORCH_FUSEDwith standard betas and epsilon. - Scheduler: Cosine learning rate scheduler with a
0.1warmup ratio. - Epochs: Trained for
5.0epochs.
This configuration suggests a focused training regimen aimed at adapting the base Qwen3-32B model to the specific characteristics of the Stack Exchange and overflow sandboxes dataset. The model leverages a substantial context window of 32768 tokens, making it suitable for processing and generating extensive technical discussions.