Rumiii/Qwen2.5-0.5B-Med-Pre-Trained-92k
Rumiii/Qwen2.5-0.5B-Med-Pre-Trained-92k is a 0.5 billion parameter Qwen2.5 base model, continued pre-trained by Rumiii on 92,000 English PubMed biomedical abstracts. This model is specifically adapted for biomedical vocabulary and medical terminology, serving as a domain-adapted foundation for further fine-tuning on medical instruction datasets. It is optimized for research into biomedical domain adaptation and as a starting point for medical reasoning models.
Loading preview...
Model Overview
This model, Rumiii/Qwen2.5-0.5B-Med-Pre-Trained-92k, is a continued pre-trained (CPT) version of the Qwen2.5-0.5B base model. Developed by Rumi Iqbal Sufi, it has undergone full-parameter training on 92,000 English PubMed biomedical abstracts, adapting its vocabulary and structure to the medical domain.
Key Characteristics
- Base Model: Built upon Qwen/Qwen2.5-0.5B.
- Domain Adaptation: All 494 million parameters were updated during training to adapt the model to biomedical vocabulary, PubMed abstract structure, and medical terminology.
- Training Data: Trained on approximately 23.6 million tokens from the VietAI/vi_pubmed dataset (English abstracts).
- Training Objective: Causal Language Modeling (CLM).
Intended Use Cases
This model is designed as a foundational component for specialized medical AI applications:
- Downstream Medical SFT: Ideal as a base model for further supervised fine-tuning (SFT) on medical instruction datasets.
- Biomedical Research: Suitable for research focused on biomedical domain adaptation.
- Medical Reasoning: Serves as a starting point for developing more advanced medical reasoning models.
Limitations
It is crucial to note that this is a base model and not instruction-tuned or a chat model. Therefore, it is not intended for direct conversational use, clinical decision-making, or patient-facing applications without further fine-tuning and validation.