Overview

DCAgent/d1_mix_top4_seq_glm47 is an 8 billion parameter language model, fine-tuned from the base Qwen/Qwen3-8B architecture. This model was specifically trained on the /e/scratch/jureap59/raoof1/sft_data/hf_hub/datasets--DCAgent--d1_mix_top4_seq_glm47_traces/snapshots/74afde1ec3e5cdfb2d579360e6e17fe4bc6b0ec7_thinking_preprocessed dataset. The fine-tuning process utilized a learning rate of 4e-05, a batch size of 1, and ran for 7 epochs, employing a cosine learning rate scheduler with a 0.1 warmup ratio. It supports a context length of 32768 tokens.

Key Capabilities

• Specialized Fine-tuning: Trained on a unique dataset (d1_mix_top4_seq_glm47_traces), suggesting potential specialization in tasks related to sequence generation, trace analysis, or specific data pattern recognition.
• Qwen3-8B Foundation: Benefits from the robust capabilities of the Qwen3-8B base model, providing a strong general language understanding and generation foundation.
• Extended Context Window: Features a 32768 token context length, enabling the processing and understanding of longer inputs and complex sequences.

Training Details

The model was trained using the following key hyperparameters:

• Learning Rate: 4e-05
• Optimizer: AdamW (torch fused) with betas=(0.9, 0.98) and epsilon=1e-08
• Epochs: 7.0
• Batch Size: 1 (per device), 16 (total distributed)

Intended Uses

This model is likely suitable for applications requiring detailed analysis or generation based on sequential data, particularly those aligned with the d1_mix_top4_seq_glm47_traces dataset's characteristics. Its extended context window makes it valuable for tasks involving long-range dependencies.