hamilton65/MMed-Llama-3-8B-EnIns

MMed-Llama-3-8B-EnIns: English Instruction-Tuned Medical LLM

MMed-Llama-3-8B-EnIns is an 8 billion parameter model built upon the Llama 3 architecture, developed by Pengcheng Qiu et al. (MAGIC-AI4Med). It is a specialized variant of the MMed-Llama 3-8B, further fine-tuned on an English instruction dataset derived from PMC-LLaMA, primarily focusing on medical question-answering (QA) tasks.

Key Capabilities & Differentiators

• Medical Domain Expertise: Specifically trained and fine-tuned for medical applications, demonstrating strong performance on medical benchmarks.
• English Instruction Following: Optimized for understanding and responding to English instructions, particularly in QA formats.
• Benchmark Performance: Achieves an average score of 72.59% across a suite of English medical benchmarks (MedQA, MedMCQA, PubMedQA, and MMLU medical subcategories), outperforming Llama 3 8B and other comparable models like MedAlpaca, PMC-LLaMA, and Mistral 7B.
• Focus on QA: While based on a multilingual foundation, this specific instruction-tuned version is noted to have limited multilingual response ability due to its English-centric SFT dataset.

Good for

• Medical Question Answering: Ideal for applications requiring accurate answers to medical questions.
• English Medical Benchmarking: Suitable for researchers and developers evaluating LLM performance in the English medical domain.
• Specialized Medical Chatbots: Can serve as a core component for chatbots focused on medical information retrieval and interaction in English.