brucewayne0459/OpenBioLLm-Derm

OpenBioLLm-Derm: Specialized Dermatology LLM

OpenBioLLm-Derm is an 8 billion parameter language model developed by Bruce_Wayne(The Batman), fine-tuned from the LLaMA-based aaditya/Llama3-OpenBioLLM-8B. This model is specifically designed for applications in dermatology, leveraging its training on a dedicated dataset of skin diseases and care.

Key Capabilities

• Dermatological Information: Provides clear, accurate, and helpful information on various skin diseases, skincare routines, treatments, and general dermatological advice.
• Chatbot Integration: Primarily intended for use as the core intelligence for dermatology chatbots.
• Specialized Knowledge: Offers in-depth knowledge within the dermatology domain, distinguishing it from general-purpose LLMs.

Training Details

The model was fine-tuned on a dataset focused on skin diseases and care (brucewayne0459/Skin_diseases_and_care). Training involved a per-device batch size of 2, 4 gradient accumulation steps, 120 max steps, and a learning rate of 2e-4 using the AdamW (8-bit) optimizer. The training was performed on a Tesla T4 GPU with 4-bit quantization and gradient checkpointing.

Limitations and Recommendations

Users should be aware that the model's responses are not a substitute for professional medical advice. It may have limitations in understanding rare conditions or those not well-represented in its training data. Further fine-tuning is recommended to enhance accuracy.