Rumiii/Qwen2.5-0.5B-Med-Post-Trained-92k
Rumiii/Qwen2.5-0.5B-Med-Post-Trained-92k is a 0.5 billion parameter Qwen2.5-based language model developed by Rumiii, specifically adapted for biomedical applications. This model underwent full-parameter continued pre-training on 92k English biomedical abstracts, followed by supervised fine-tuning on a general instruction dataset. It is optimized for medical question answering and clinical education, serving as a lightweight solution for medical AI prototyping and research into small biomedical language models.
Loading preview...
Model Overview
Rumiii/Qwen2.5-0.5B-Med-Post-Trained-92k is a 0.5 billion parameter language model built upon the Qwen2.5-0.5B architecture. It has been specialized for the medical domain through a two-stage training process: full-parameter continued pre-training (CPT) on biomedical text and subsequent supervised fine-tuning (SFT) on a general instruction dataset.
Key Capabilities & Training Details
- Domain Adaptation: The model underwent CPT on the
VietAI/vi_pubmeddataset, comprising 92,000 English biomedical abstracts (approximately 23.6 million tokens), enhancing its understanding of medical terminology and concepts. - Instruction Following: Following CPT, it was fine-tuned using the
causal-lm/ultrachatdataset (20,000 samples) to improve its ability to follow instructions and generate coherent responses, utilizing the Qwen2.5 ChatML chat template. - Lightweight & Efficient: As a 0.5B parameter model, it is designed for efficient deployment and research, particularly suitable for consumer hardware.
- Recommended Usage: Requires a system prompt for optimal performance, acting as a knowledgeable medical AI assistant named MedAssist.
Intended Use Cases
- Medical Question Answering: Providing informative answers to medical queries.
- Clinical Education: Supporting learning and understanding in clinical contexts.
- Biomedical Research: Investigating the performance and capabilities of small language models in the biomedical field.
- AI Prototyping: Developing lightweight medical AI applications.
Limitations & Considerations
- Reasoning Depth: Due to its 0.5B parameter size, its reasoning capabilities are limited compared to larger models.
- Clinical Accuracy: Outputs are not guaranteed to be clinically accurate and require verification by medical professionals; it is not intended for clinical decision-making.
- Greeting Inconsistency: May produce inconsistent responses to simple greetings; it performs best with direct questions.
- Language Support: English only.