starmpcc/Asclepius-13B
Asclepius-13B is a 13 billion parameter clinical Large Language Model (LLM) developed by starmpcc, fine-tuned from LLaMA-13B. It is the first publicly shareable clinical LLM trained with synthetic data, specializing in clinical NLP tasks such as Named Entity Recognition, Summarization, and Question Answering. This model is designed for research purposes in processing clinical notes and related medical text.
Loading preview...
Asclepius-13B: A Clinical Large Language Model
Asclepius-13B is a 13 billion parameter clinical Large Language Model (LLM) developed by starmpcc, fine-tuned from LLaMA-13B. It stands out as the first publicly shareable clinical LLM, uniquely trained using synthetic clinical notes and further fine-tuned with clinical instruction-response pairs. A variant, Asclepius-R, trained on MIMIC-III discharge summaries, is also available on Physionet.
Key Capabilities
Asclepius-13B is designed to perform a range of clinical Natural Language Processing (NLP) tasks on clinical notes, including:
- Named Entity Recognition
- Abbreviation Expansion
- Relation Extraction
- Temporal Information Extraction
- Coreference Resolution
- Paraphrasing
- Summarization
- Question Answering
Training Details
The model underwent initial causal language modeling on synthetic clinical notes, followed by fine-tuning with clinical instruction-response pairs. The training process involved significant computational resources, with pre-training taking 1 hour 52 minutes and instruction fine-tuning taking 12 hours 16 minutes, both utilizing 8x A100 80G GPUs.
Good For
- Research in Clinical NLP: Ideal for academic and research applications focusing on medical text analysis.
- Developing Clinical AI Tools: Can serve as a foundational model for building applications that require understanding and processing clinical documentation.
Note: This model is intended for research purposes only, as specified by its CC-BY-NC-SA 4.0 license.