penfever/GLM-4_6-taskmaster2-32eps-32k-fixeps
The penfever/GLM-4_6-taskmaster2-32eps-32k-fixeps model is an 8 billion parameter language model, fine-tuned from Qwen/Qwen3-8B. It features a 32,768 token context length and was trained for 7 epochs on the penfever/GLM-4.6-taskmaster2-32eps-32k dataset. This model is optimized for tasks related to its specific fine-tuning dataset, making it suitable for applications requiring specialized understanding based on that data.
Loading preview...
Model Overview
This model, penfever/GLM-4_6-taskmaster2-32eps-32k-fixeps, is an 8 billion parameter language model derived from the Qwen3-8B architecture. It has been fine-tuned specifically on the penfever/GLM-4.6-taskmaster2-32eps-32k dataset, indicating a specialization towards tasks represented within that dataset. The model supports a substantial context length of 32,768 tokens, allowing for processing and generating longer sequences of text.
Training Details
The fine-tuning process involved 7 epochs with a learning rate of 4e-05. Key hyperparameters included a train_batch_size of 1 and an eval_batch_size of 8, utilizing a distributed training setup across 16 devices. The optimizer used was ADAMW_TORCH_FUSED with specific beta and epsilon values, and a cosine learning rate scheduler with a warmup ratio of 0.1.
Intended Use Cases
Given its fine-tuning on a specific dataset, this model is best suited for applications that align with the characteristics and content of the penfever/GLM-4.6-taskmaster2-32eps-32k dataset. Developers should consider its specialized training for tasks where this particular data distribution is relevant.