Internist.ai 7b: A Medical Domain LLM

Internist.ai 7b is a 7 billion parameter large language model developed by UCLouvain and Cliniques Universitaires Saint-Luc, specifically designed for the medical domain. It is fine-tuned from Mistral-7B-v0.1 and notable for being the first 7B model to achieve a passing score of 60% on the MedQA (USMLE) benchmark, demonstrating the effectiveness of a "physician-in-the-loop" training approach.

Key Capabilities & Performance

• Medical Expertise: Achieves a 60.5% score on MedQA, outperforming other 7B medical models like PMC LLaMA 7b and Meditron 7b.
• Strong Medical Benchmarks: Shows superior performance across various medical evaluations, including PubMedQA (79.4%), MMLU Professional Medicine (76.1%), and MMLU Clinical Knowledge (70.6%).
• High-Quality Training Data: Trained on a curated dataset including OpenOrca-GPT4, 11,332 articles from UpToDate, 10,376 medical textbooks, and 400M tokens of synthetic data, all guided by medical doctors.
• Retained General Capabilities: The training methodology aimed to retain general domain capabilities while specializing in medical knowledge.

Intended Use Cases

• Medical Professional Assistant: Designed to serve as an assistant for medical professionals, aiding in clinical decision support or documentation.
• Proof of Concept: Serves as a proof of concept for the benefits of high-quality, physician-curated medical literature in LLM training.

Important Advisory

• Physician-in-the-Loop: The model was designed by medical doctors for medical doctors. It has not undergone specific training for use by non-medical professionals.
• No Live Environment Use: Not recommended for use in a live environment without extensive evaluation through prospective clinical trials and additional safety training.
• No Instruction Tuning for Safety: The model was not specifically instruction-tuned to ensure safety, emphasizing the need for professional oversight.