laion/exp-gfi-staqc-askllm-filtered-10K_glm_4_7_traces_jupiter_cleaned
The laion/exp-gfi-staqc-askllm-filtered-10K_glm_4_7_traces_jupiter_cleaned model is an 8 billion parameter language model, fine-tuned from Qwen/Qwen3-8B. It was trained on the /data/cat/ws/befe330h-befe330h-otagent/huggingface/hub/datasets--DCAgent--exp-gfi-staqc-askllm-filtered-10K_glm_4.7_traces_jupiter_cleaned/snapshots/6b97b7d8d3baf1c49d01353ea2647df88727ddd8_thinking_preprocessed dataset. This model is designed for specific applications related to its fine-tuning data, offering a 32768 token context length.
Loading preview...
Model Overview
This model, laion/exp-gfi-staqc-askllm-filtered-10K_glm_4_7_traces_jupiter_cleaned, is an 8 billion parameter language model derived from the Qwen/Qwen3-8B architecture. It has been specifically fine-tuned on a unique dataset, /data/cat/ws/befe330h-befe330h-otagent/huggingface/hub/datasets--DCAgent--exp-gfi-staqc-askllm-filtered-10K_glm_4.7_traces_jupiter_cleaned/snapshots/6b97b7d8d3baf1c49d01353ea2647df88727ddd8_thinking_preprocessed, suggesting a specialization for tasks related to the content of this training data.
Key Training Details
The fine-tuning process involved the following hyperparameters:
- Learning Rate: 4e-05
- Batch Size: 1 (train), 8 (eval)
- Gradient Accumulation Steps: 2, leading to a total train batch size of 16
- Optimizer: ADAMW_TORCH_FUSED
- LR Scheduler: Cosine type with a 0.1 warmup ratio
- Epochs: 7.0
The model was trained using Transformers 4.57.6, Pytorch 2.9.0+cu128, Datasets 4.4.1, and Tokenizers 0.22.2.
Intended Use
Given its fine-tuning on a specific dataset, this model is likely intended for applications that align with the characteristics and domain of its training data. Users should consider the nature of the exp-gfi-staqc-askllm-filtered-10K_glm_4.7_traces_jupiter_cleaned dataset when evaluating its suitability for their specific use cases.