guangyangnlp/Qwen3-4B-SFT-medical-1e-5
The guangyangnlp/Qwen3-4B-SFT-medical-1e-5 is a 4 billion parameter language model, fine-tuned from the Qwen/Qwen3-4B architecture. This model is specifically specialized for medical applications, having been fine-tuned on the medical_o1_train dataset. It is designed to perform well in medical contexts, leveraging its base Qwen3 architecture with a 32K context length.
Loading preview...
Model Overview
The guangyangnlp/Qwen3-4B-SFT-medical-1e-5 is a 4 billion parameter language model derived from the Qwen/Qwen3-4B base model. It has been specifically fine-tuned on the medical_o1_train dataset, indicating its specialization for medical-related natural language processing tasks. The model was trained with a learning rate of 1e-05 over 3 epochs, utilizing an AdamW optimizer and a cosine learning rate scheduler.
Key Characteristics
- Base Model: Qwen/Qwen3-4B
- Parameter Count: 4 billion
- Context Length: 32,768 tokens (inherited from base model)
- Specialization: Fine-tuned for medical applications using the
medical_o1_traindataset. - Training Hyperparameters: Employed a learning rate of 1e-05, a batch size of 4 (total effective batch size of 128 with gradient accumulation), and trained for 3 epochs.
Intended Use Cases
This model is primarily intended for use in scenarios requiring medical domain understanding and generation. Its fine-tuning on a medical dataset suggests suitability for tasks such as:
- Processing and understanding medical texts.
- Generating responses or summaries related to medical information.
- Assisting in medical information retrieval or question-answering within a medical context.
Limitations
The model's specific limitations and full intended uses are not extensively detailed in the provided information, suggesting further evaluation and understanding of its performance on diverse medical tasks would be beneficial.