starmpcc/Asclepius-Mistral-7B-v0.3
Asclepius-Mistral-7B-v0.3 by starmpcc is a 7 billion parameter clinical Large Language Model, fine-tuned from Mistral-7B-v0.3 with an increased context length of 8192 tokens. This model specializes in clinical natural language processing tasks, including Named Entity Recognition, summarization, and question answering on clinical notes. It is designed for research purposes in clinical NLP, leveraging synthetic clinical notes for its training.
Loading preview...
Asclepius-Mistral-7B-v0.3: A Clinical LLM
Asclepius-Mistral-7B-v0.3 is a 7 billion parameter clinical Large Language Model developed by starmpcc. It is an enhanced version of Asclepius-7B, built upon the Mistral-7B-v0.3 base model, featuring an extended maximum sequence length of 8192 tokens. The model is specifically designed for clinical natural language processing tasks.
Key Capabilities
This model is proficient in performing various clinical NLP tasks using clinical notes, including:
- Named Entity Recognition
- Abbreviation Expansion
- Relation Extraction
- Temporal Information Extraction
- Coreference Resolution
- Paraphrasing
- Summarization
- Question Answering
Training Details
The model was initially trained using causal language modeling on a dataset of synthetic clinical notes provided by starmpcc. Subsequently, it underwent fine-tuning with clinical instruction-response pairs. The training procedure involved pre-training for approximately 3 hours and instruction fine-tuning for over 30 hours, both utilizing 4x A100 80G GPUs.
Intended Use
Asclepius-Mistral-7B-v0.3 is intended solely for research purposes in the domain of clinical NLP. Its license is CC-BY-NC-SA 4.0.