CompassioninMachineLearning/Qwen3-8b-compassion-cleaned-10k-CPT-best-step750
Qwen3-8b-compassion-cleaned-10k-CPT-best-step750 is an 8 billion parameter language model based on the Qwen3 architecture, developed by CompassioninMachineLearning. This model is a merged bf16 version derived from a specific checkpoint (step 750) of a fine-tuning run, chosen for its lowest validation loss of 1.5582. It is primarily intended for applications requiring a Qwen3-based model optimized through continued pre-training on a specific dataset, indicated by "compassion-cleaned-10k-CPT".
Loading preview...
Overview
Qwen3-8b-compassion-cleaned-10k-CPT-best-step750 is an 8 billion parameter language model developed by CompassioninMachineLearning. This model is a merged bf16 version, specifically derived from checkpoint-750 of the CompassioninMachineLearning/Qwen3-8b-compassion-cleaned-10k-20260910-CPT-LoRA-checkpoints training run. This particular checkpoint was selected because it exhibited the lowest validation loss (1.5582) during its training process, which continued up to step 1500.
Key Characteristics
- Architecture: Based on the Qwen3 model family.
- Parameter Count: 8 billion parameters.
- Precision: Merged in bf16 format.
- Optimization: Represents the best performing checkpoint (step 750) from a continued pre-training (CPT) run, identified by its lowest validation loss.
Good For
- Developers seeking a Qwen3-8b base model that has undergone additional domain-specific pre-training, as indicated by the "compassion-cleaned-10k-CPT" in its name.
- Applications where a model optimized for a specific dataset, achieving the lowest validation loss during its training phase, is beneficial.