Lowenzahn/KoBioMed-Llama-3.1-8B

KoBioMed-Llama-3.1-8B: A Specialized Bilingual Biomedical LLM

KoBioMed-Llama-3.1-8B, developed by ezCaretech's AI Team, is an 8 billion parameter generative language model designed specifically for the biomedical domain. It supports both English and Korean languages, making it a valuable resource for a broad range of research and application areas.

Key Capabilities and Features

• Bilingual Specialization: Continuously pre-trained on a large dataset of PubMed abstracts and their Korean translations, ensuring deep understanding of biomedical concepts in both languages.
• Strong Performance: Achieves state-of-the-art results on various Korean and English biomedical benchmarks, including KMMLU, KorMedMCQA, MedMCQA, MMLU, and PubMedQA. For instance, it scored 0.4010 on KMMLU and 0.7800 on PubMedQA, outperforming Llama-3.1-8B in several key metrics.
• Pre-trained Model: This is a pre-trained model, serving as an excellent foundation for further post-training, such as instruction tuning, to adapt it to specific tasks.
• Robust Training Data: Trained on preprocessed PubMed abstracts from 2000-2023, with extensive cleansing, de-duplication, and quality filtering.
• Context Length: Features a context length of 8,192 tokens.

Use Cases and Limitations

This model is particularly well-suited for applications within the biomedical and medical research community. However, users should be aware of its limitations, including the potential for generating biased or outdated information, and its performance may degrade on tasks outside the biomedical and healthcare domains. Critical information generated by the model should always be independently verified.