Asclepius-13B: A Clinical Large Language Model
Asclepius-13B is a 13 billion parameter clinical Large Language Model (LLM) developed by starmpcc, fine-tuned from LLaMA-13B. It stands out as the first publicly shareable clinical LLM, uniquely trained using synthetic clinical notes and further fine-tuned with clinical instruction-response pairs. A variant, Asclepius-R, trained on MIMIC-III discharge summaries, is also available on Physionet.
Key Capabilities
Asclepius-13B is designed to perform a range of clinical Natural Language Processing (NLP) tasks on clinical notes, including:
- Named Entity Recognition
- Abbreviation Expansion
- Relation Extraction
- Temporal Information Extraction
- Coreference Resolution
- Paraphrasing
- Summarization
- Question Answering
Training Details
The model underwent initial causal language modeling on synthetic clinical notes, followed by fine-tuning with clinical instruction-response pairs. The training process involved significant computational resources, with pre-training taking 1 hour 52 minutes and instruction fine-tuning taking 12 hours 16 minutes, both utilizing 8x A100 80G GPUs.
Good For
- Research in Clinical NLP: Ideal for academic and research applications focusing on medical text analysis.
- Developing Clinical AI Tools: Can serve as a foundational model for building applications that require understanding and processing clinical documentation.
Note: This model is intended for research purposes only, as specified by its CC-BY-NC-SA 4.0 license.