chaoyi-wu/PMC_LLAMA_7B
PMC_LLAMA_7B is a 7 billion parameter LLaMA-based causal language model developed by chaoyi-wu. It has been fine-tuned on the PMC papers from the S2ORC dataset, specializing in biomedical and scientific text generation. This model is optimized for tasks requiring understanding and generation within the domain of scientific literature, particularly medical and biological research.
Loading preview...
Overview
PMC_LLAMA_7B is a 7 billion parameter language model built upon the LLaMA architecture. Developed by chaoyi-wu, its primary distinction lies in its specialized training regimen: it was fine-tuned extensively on the Public Medical Central (PMC) papers found within the S2ORC dataset. This focused training aims to enhance its performance and relevance for tasks within the biomedical and scientific domains.
Key Capabilities
- Specialized Domain Knowledge: Excels in generating and understanding text related to medical, biological, and scientific research, owing to its fine-tuning on PMC papers.
- LLaMA Architecture: Benefits from the robust base architecture of LLaMA, providing a strong foundation for language understanding and generation.
Training Details
The model underwent 5 epochs of fine-tuning with a batch size of 128 and a cutoff length of 512 tokens. A learning rate of 2e-5 was used, with 512 tokens sampled per paper during each epoch of training. This targeted approach allows PMC_LLAMA_7B to capture the nuances and terminology prevalent in scientific literature.
Good For
- Generating scientific abstracts or summaries.
- Assisting with literature review in medical or biological fields.
- Answering questions based on scientific papers.
- Developing applications that require domain-specific language understanding in biomedicine.