winninghealth/WiNGPT2-7B-Base

WiNGPT2-7B-Base: A Specialized Medical LLM

WiNGPT2-7B-Base is a 7 billion parameter large language model developed by winninghealth, built upon the Qwen-7B architecture. Initiated in January 2023, the project focuses on integrating professional medical knowledge and data to enhance healthcare services. The model incorporates advanced architectural features like RoPE relative position encoding, SwiGLU activation, and RMSNorm.

Key Capabilities

• Medical Knowledge Q&A: Answers questions across various medical and health topics, including symptoms, treatments, drugs, prevention, and examinations.
• Natural Language Understanding: Processes medical terminology and clinical text for key information extraction and categorization.
• Multi-turn Dialogue: Engages in contextual, accurate conversations, capable of role-playing various medical professionals.
• Multi-task Support: Supports 32 medical tasks across 8 major and 18 sub-scenarios.

Training and Performance

The model was trained on an extensive medical corpus, including 15,000 drug instructions, 9,720 disease knowledge items, 300 medical books, 1.1 million inspection/test records, and 2.56 million medical knowledge graph triplets. It also utilized 50,000 human-annotated instructions and 5 million medical Q&A dialogues. On the MedQA-MCMLE benchmark, WiNGPT2-7B-Base achieved an impressive 82.3% average score, significantly outperforming other models in its class, highlighting its strong specialization in medical domains.

Good for

• Medical Q&A systems: Providing accurate answers to patient and professional medical queries.
• Diagnostic support: Offering reference information for diagnosis and treatment planning.
• Medical text analysis: Extracting and categorizing information from clinical records and medical literature.
• AI doctor simulations: Engaging in realistic, multi-turn medical conversations.