Jerry999/Monitorability-CTPTR
Jerry999/Monitorability-CTPTR is a 9 billion parameter Qwen3.5-based causal language model that has undergone continued pre-training on the cpt_mixed dataset. This model is specifically adapted from Qwen/Qwen3.5-9B, focusing on enhancing its capabilities through further training. It is designed for applications requiring a Qwen3.5 architecture with additional domain-specific or general knowledge acquired during its continued pre-training phase. Its primary strength lies in leveraging the foundational Qwen3.5 model with an expanded training regimen.
Loading preview...
Overview
Jerry999/Monitorability-CTPTR is a 9 billion parameter language model derived from the Qwen/Qwen3.5-9B architecture. It has undergone a process of continued pre-training (CPT) on a dataset referred to as cpt_mixed.
Key Training Details
This model was trained using specific hyperparameters to optimize its performance during the continued pre-training phase:
- Learning Rate: 1e-05
- Optimizer: ADAMW_TORCH with standard betas and epsilon
- Scheduler: Constant with 20 warmup steps
- Training Steps: 9500
- Batch Size: A total training batch size of 128 was achieved with a
train_batch_sizeof 1,gradient_accumulation_stepsof 4, and 32 devices.
Intended Use Cases
While specific intended uses and limitations are not detailed in the provided information, as a continued pre-training version of Qwen3.5-9B, it is generally suitable for tasks where the base Qwen3.5 model performs well, with potential improvements or adaptations due to the additional training data. Developers should consider its 9B parameter size and 32768 token context length for applications requiring a balance of performance and efficiency.