Model Overview

This model, laion/exp-uns-tezos-10x_glm_4_7_traces_jupiter, is an 8 billion parameter language model derived from the Qwen/Qwen3-8B architecture. It has undergone a specific fine-tuning process using a unique dataset: /data/cat/ws/befe330h-befe330h-otagent/huggingface/hub/datasets--DCAgent--exp-uns-tezos-10x_glm_4.7_traces_jupiter/snapshots/c8f326b977ebe6dbabb49aa3145fd991bd7753fe_thinking_preprocessed.

Training Details

The fine-tuning process utilized the following key hyperparameters:

• Learning Rate: 4e-05
• Batch Sizes: train_batch_size of 1, eval_batch_size of 8, leading to a total_train_batch_size of 16 and total_eval_batch_size of 64 (with 8 devices and 2 gradient accumulation steps).
• Optimizer: ADAMW_TORCH_FUSED with betas=(0.9, 0.98) and epsilon=1e-08.
• Scheduler: Cosine learning rate scheduler with a 0.1 warmup ratio.
• Epochs: Trained for 7.0 epochs.

Intended Use

Given the highly specific training dataset, this model is likely intended for specialized applications related to the nature of the exp-uns-tezos-10x_glm_4.7_traces_jupiter data. Developers should consider its unique fine-tuning for tasks that align with this specific domain rather than general-purpose language generation.