Palmyra-Med-70B-32K: A Specialized LLM for Healthcare
Palmyra-Med-70B-32K, developed by Writer, is a 70 billion parameter large language model with a 32,768 token context window, meticulously designed for the healthcare and biomedical sectors. It builds upon the Palmyra-Med-70B foundation, incorporating an extended context length and specialized fine-tuning on extensive, high-quality biomedical data.
Key Capabilities and Performance
- Biomedical Expertise: Achieves an average score of 85.87% on biomedical benchmarks, outperforming models like GPT-4, Claude Opus, Gemini, and Med-PaLM-2 base model.
- Extended Context: Features a 32K context window, demonstrating robust capability in processing extensive medical documents, as evidenced by near-perfect scores in needle-in-haystack evaluations.
- Advanced Fine-tuning: Utilizes Direct Preference Optimization (DPO) and a custom, diverse medical instruction dataset to enhance domain-specific accuracy and fluency.
- Clinical Data Analysis: Excels in analyzing and summarizing complex clinical notes, EHR data, and discharge summaries, extracting key information and performing advanced clinical entity recognition.
Intended Use Cases
Palmyra-Med-70B-32K is intended for non-commercial and research use in English, particularly for assistant-like chat and various natural language generation tasks within the medical field. It supports applications such as clinical decision support, pharmacovigilance, and medical research by enhancing information retrieval and knowledge discovery from biomedical sources.
Important Considerations
- The model may contain inaccuracies or biases and has not been rigorously evaluated in clinical trials.
- It is not intended for direct patient care, clinical decision support, or professional medical purposes and should not replace professional medical judgment.
- Use is restricted by Writer's Acceptable Use Policy and the Writer open model license.