Henrychur/MMed-Llama-3-8B

MMed-Llama-3-8B: A Multilingual Medical Foundation Model

MMed-Llama-3-8B is an 8 billion parameter language model developed by Henrychur, specifically designed for the medical domain. It is built upon the Llama 3 architecture and has undergone extensive further pretraining on MMedC, a large-scale multilingual medical corpus containing 25.5 billion tokens.

Key Capabilities & Features

• Multilingual Medical Expertise: Enhanced understanding and generation of medical text across various languages due to specialized pretraining on MMedC.
• Foundation Model: Provides a strong base for medical NLP tasks, though it has not been instruction fine-tuned.
• Competitive Performance: The MMedLM series, including this model, has demonstrated superior performance compared to other open-source models on the MMedBench multilingual medical question-answering benchmark, even rivaling GPT-4 in some medical contexts.
• Comprehensive Training: Further pretrained for 15,000 iterations with a global batch size of 512 and a cutoff length of 8192, optimizing its medical domain knowledge.

Use Cases & Considerations

This model is ideal for researchers and developers building applications that require deep medical domain understanding in a multilingual setting. As a foundation model, it is best suited for further fine-tuning on specific downstream medical tasks. Users should note that it is not instruction-tuned out-of-the-box, meaning it will require additional fine-tuning for conversational or instruction-following applications.