winninghealth/WiNGPT2-7B-Chat
WiNGPT2-7B-Chat is a 7 billion parameter instruction-tuned causal language model developed by winninghealth, based on the Transformer architecture and Qwen-7B. It is specifically designed for the medical vertical domain, excelling in medical knowledge Q&A, natural language understanding of medical texts, and multi-turn dialogues in healthcare scenarios. The model supports 32 medical tasks across eight major medical scenarios, aiming to provide intelligent medical information services.
Loading preview...
WiNGPT2-7B-Chat: A Specialized Medical LLM
WiNGPT2-7B-Chat, developed by winninghealth, is a 7 billion parameter large language model built upon the Transformer architecture, utilizing RoPE relative position encoding, SwiGLU activation, and RMSNorm. It leverages Qwen-7B as its foundational pre-trained model and has been extensively fine-tuned on a massive medical corpus, including 15,000 drug instructions, 9,720 disease knowledge entries, 300 medical textbooks, 1.1 million inspection/test knowledge entries, and 2.56 million medical knowledge graph triplets.
Key Capabilities
- Medical Knowledge Q&A: Answers questions on symptoms, treatments, drugs, prevention, and examinations.
- Natural Language Understanding: Comprehends medical terminology and patient records, extracting and categorizing key information.
- Multi-turn Dialogue: Engages in contextual conversations, acting as various medical professionals.
- Multi-task Support: Supports 32 medical tasks across 8 major medical scenarios and 18 sub-scenarios.
- Retrieval-Augmented Generation: Incorporates enhanced retrieval capabilities for more accurate responses.
Performance Highlights
On the Chinese medical professional assessment MedQA-MCMLE (Zero-shot), WiNGPT2-7B-Base achieved an impressive 82.3% average score, significantly outperforming other models in its class, including Baichuan2-13B-Base (62.9%) and Qwen-7B (59.3%). This demonstrates its high accuracy and specialized knowledge in the medical domain.
Good For
- Medical Q&A systems: Providing accurate answers to health-related queries.
- Diagnostic support: Offering reference information for medical professionals.
- Medical record analysis: Extracting key information from clinical texts.
- Patient interaction: Developing AI-driven virtual assistants for healthcare.
Limitations
WiNGPT2 provides information and suggestions for reference only and should not replace professional medical advice, diagnosis, or treatment. Users are advised to consult medical professionals and independently evaluate the information provided, as the model's information may contain errors or inaccuracies.