dmis-lab/self-biorag-7b-olaph
The dmis-lab/self-biorag-7b-olaph model is a 7 billion parameter language model fine-tuned from Minbyul/selfbiorag-7b-wo-kqa_golden-iter-dpo-step3-filtered. It was trained on the HuggingFace MedLFQA dataset (without kqa_golden) with a context length of 4096 tokens. This model is specialized for biomedical question answering tasks, demonstrating improved reward metrics on its evaluation set. Its primary strength lies in processing and generating responses related to medical and life sciences information.
Loading preview...
Model Overview
The dmis-lab/self-biorag-7b-olaph is a 7 billion parameter language model developed by dmis-lab. It is a fine-tuned iteration of Minbyul/selfbiorag-7b-wo-kqa_golden-iter-dpo-step3-filtered, specifically adapted for biomedical language understanding and generation tasks.
Key Characteristics
- Base Model: Fine-tuned from
Minbyul/selfbiorag-7b-wo-kqa_golden-iter-dpo-step3-filtered. - Training Data: Utilizes the HuggingFace MedLFQA dataset (excluding
kqa_golden) for specialized domain adaptation. - Performance Metrics: Achieved a reward accuracy of 0.6319 and a chosen reward of -0.0828 on its evaluation set, indicating its performance in distinguishing preferred responses.
- Training Configuration: Trained with a learning rate of 5e-07, a total batch size of 64, and a cosine learning rate scheduler over 1 epoch.
Intended Use Cases
This model is particularly suited for applications requiring nuanced understanding and generation within the biomedical domain. While specific intended uses are not fully detailed, its training on MedLFQA suggests applicability in:
- Biomedical Question Answering: Answering queries related to medical literature, biological concepts, and clinical information.
- Information Retrieval: Assisting in extracting relevant information from large biomedical text corpora.
Further details on specific use cases and limitations are pending from the model developers.