laion/exp-psu-swesmith-1K_glm_4-7_traces_jupiter__0-08__Qwen3-8B
The laion/exp-psu-swesmith-1K_glm_4-7_traces_jupiter__0-08__Qwen3-8B model is an 8 billion parameter language model, fine-tuned from the Qwen/Qwen3-8B architecture. It was specifically trained on the /e/data1/datasets/playground/ot/hf_hub/datasets--DCAgent--exp-psu-swesmith-1K_glm_4.7_traces_jupiter/snapshots/24c8342833108c3a15a23b64f37b83ff7e65efa4_thinking_preprocessed dataset. This model is a specialized fine-tune, indicating potential optimization for tasks related to the specific training data, and supports a context length of 32768 tokens.
Loading preview...
Model Overview
This model, laion/exp-psu-swesmith-1K_glm_4-7_traces_jupiter__0-08__Qwen3-8B, is an 8 billion parameter language model derived from the Qwen/Qwen3-8B base architecture. It has been fine-tuned on a specific dataset, /e/data1/datasets/playground/ot/hf_hub/datasets--DCAgent--exp-psu-swesmith-1K_glm_4.7_traces_jupiter/snapshots/24c8342833108c3a15a23b64f37b83ff7e65efa4_thinking_preprocessed, suggesting a specialization for tasks aligned with the characteristics of this training data.
Training Details
The fine-tuning process involved several key hyperparameters:
- Learning Rate: 4e-05
- Batch Size: 1 (train), 8 (eval)
- Gradient Accumulation: 3 steps
- Optimizer: ADAMW_TORCH_FUSED with betas=(0.9, 0.98) and epsilon=1e-08
- LR Scheduler: Cosine type with a warmup ratio of 0.1
- Epochs: 7.0
- Devices: Trained across 32 GPUs
This configuration indicates a thorough fine-tuning approach designed to adapt the base Qwen3-8B model to the nuances of the target dataset. The model supports a context length of 32768 tokens.