Rishidar/autoscientist-healthcare-qlora
Rishidar/autoscientist-healthcare-qlora is a 0.5 billion parameter causal language model fine-tuned from Qwen/Qwen2.5-0.5B-Instruct. It was trained using QLoRA on a healthcare-adapted dataset from the Adaption Labs AutoScientist Challenge. This model is specifically optimized for healthcare-related natural language processing tasks, leveraging its 32768 token context length for specialized applications.
Loading preview...
AutoScientist Healthcare Model Overview
Rishidar/autoscientist-healthcare-qlora is a specialized language model developed for the AutoScientist Competition, focusing on healthcare applications. It is a 0.5 billion parameter model, fine-tuned from the robust Qwen/Qwen2.5-0.5B-Instruct base model.
Key Capabilities & Training
- Healthcare Specialization: The model has been specifically adapted for healthcare contexts, utilizing a high-quality, Grade A dataset from Adaption Labs' AutoScientist Challenge.
- Efficient Fine-tuning: Training was conducted using QLoRA (Quantized Low-Rank Adaptation) with 4-bit NF4 quantization, an
rvalue of 32, and analphaof 64, over 3 epochs with a learning rate of 0.0002. This method allows for efficient adaptation of the base model to the target domain. - Context Length: It supports a substantial context length of 32768 tokens, enabling it to process and understand longer healthcare-related texts and queries.
Ideal Use Cases
This model is particularly well-suited for applications requiring domain-specific understanding within the healthcare sector. Its fine-tuning on a dedicated healthcare dataset makes it a strong candidate for tasks such as:
- Processing and generating healthcare-related text.
- Assisting with medical information retrieval or summarization.
- Developing specialized chatbots or assistants for healthcare contexts.