EPFLiGHT/Meditron3-Qwen2.5-7B

TEXT GENERATIONConcurrency Cost:1Model Size:7.6BQuant:FP8Ctx Length:32kTool Calling:SupportedPublished:Jan 16, 2025License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

EPFLiGHT/Meditron3-Qwen2.5-7B is a 7.6 billion parameter causal decoder-only transformer language model developed by the OpenMeditron initiative, fine-tuned from Qwen2.5-7B. This model is specialized in clinical medicine, with a focus on equitable representation, contextual diversity, and real-world evidence-based guidelines, particularly for limited-resource and humanitarian settings. It is designed as a research-only tool to enhance clinical decision-making and access to medical information. The model demonstrates improved performance on medical multiple-choice question benchmarks compared to its base model.

Loading preview...

Meditron3-Qwen2.5-7B: Specialized Medical LLM

EPFLiGHT/Meditron3-Qwen2.5-7B is a 7.6 billion parameter large language model developed by the OpenMeditron initiative. It is fine-tuned from the Qwen2.5-7B base model and is specifically designed for clinical medicine.

Key Specializations & Features

  • Medical Specialization: Focused on clinical medicine, emphasizing equitable representation, contextual diversity, and actionable real-world evidence-based guidelines.
  • Humanitarian Focus: Particular effort made to represent limited-resource and humanitarian settings, neglected populations, and diseases.
  • Training Data: Trained on a diverse mixture of expert-curated publicly available data, including clinical guidelines, peer-reviewed medical publications, synthetic differential diagnoses, replay data, and LLM-enhanced medical MCQs.
  • Research-Only Tool: Intended for research to study and evaluate the potential of LLMs in enhancing clinical decision-making and access to evidence-based medical information.
  • Foundation Model: Released as a foundation model, not instruction-tuned, but adaptable for downstream tasks via in-context learning or fine-tuning.

Performance & Evaluation

Evaluated on medical multiple-choice questions, Meditron3-Qwen2.5-7B shows improved scores across MedmcQA, MedQA, and PubmedQA compared to the Qwen/Qwen2.5-7B-Instruct base model. For instance, it achieved 55.56 on MedmcQA, a 2.32 point difference over the base model.

Important Considerations

  • Disclaimer: This model is provided "as is" and is not validated for medical use or clinical decision-making. It should not be used for self-diagnosis or treatment.
  • Static Model: This is a static model trained on an offline dataset, with future performance enhancements planned.

Finetunes
1 models