Meerkat-70B: An Advanced Medical AI System

Meerkat-70B is a 70 billion parameter instruction-tuned medical AI model developed by dmis-lab. It is built upon Meta's Llama-3-70B-Instruct and specifically enhanced for complex medical reasoning tasks.

Key Capabilities & Training

• Medical Reasoning: Fine-tuned using a unique synthetic dataset derived from 18 medical textbooks, focusing on high-quality chain-of-thought reasoning paths.
• Instruction Following: Incorporates diverse instruction-following datasets to improve generalization across various user prompts, including the AlpaCare instruction dataset (52K examples).
• Context Length: Supports an 8192-token context window, enabling processing of longer medical queries and case studies.

Performance Highlights

Meerkat-70B demonstrates strong performance across a suite of medical benchmarks, often outperforming other specialized medical LLMs and GPT-3.5:

• Overall: Achieves an average score of 77.9% across seven medical benchmarks, including MedQA, USMLE sample test, Medbullets, MedMCQA, and MMLU-Medical.
• Specific Benchmarks: Scores 82.6% on MedQA, 87.4% on USMLE, and 86.9% on MMLU-Medical (average across six medical-related subjects).

Use Cases

• Medical Question Answering: Designed to answer USMLE-style questions and other multiple-choice medical exams with step-by-step reasoning.
• Clinical Support: Capable of engaging in multi-turn dialogues to gather medical history and provide scientifically-grounded answers, acting as a helpful doctor or healthcare professional.

For more details, refer to the associated paper: Small Language Models Learn Enhanced Reasoning Skills from Medical Textbooks.