Overview

This model, laion/exp-uns-tezos-40x_glm_4_7_traces_jupiter, is an 8 billion parameter language model derived from the Qwen3-8B architecture. It has undergone fine-tuning on a specific dataset: /data/cat/ws/befe330h-befe330h-otagent/huggingface/hub/datasets--DCAgent--exp-uns-tezos-40x_glm_4.7_traces_jupiter/snapshots/f8298de8c6c58ab7239a9bae3cc91cae9525ba79_thinking_preprocessed.

Training Details

The fine-tuning process utilized a learning rate of 4e-05 with an AdamW optimizer. Training was conducted across 8 GPUs with a total batch size of 16, accumulating gradients over 2 steps. A cosine learning rate scheduler with a 0.1 warmup ratio was employed over 7 epochs. The model was trained using Transformers 4.57.6, PyTorch 2.9.0+cu128, Datasets 4.4.1, and Tokenizers 0.22.2.

Key Characteristics

• Base Model: Qwen/Qwen3-8B
• Parameter Count: 8 billion
• Context Length: 32768 tokens
• Fine-tuning: Specialized on a unique dataset, suggesting tailored performance for tasks aligned with that data.

Intended Use

Given its fine-tuning on a specific dataset, this model is best suited for applications and research that align with the characteristics and domain of the training data. Further details on specific intended uses and limitations would require more information about the fine-tuning dataset itself.