Dario213/Qwen3-4B-medical-reasoning
Dario213/Qwen3-4B-medical-reasoning is a 4 billion parameter Qwen3 model developed by Dario213, fine-tuned for medical reasoning tasks. This model leverages LoRA adapters and was trained using Unsloth and Huggingface's TRL library for accelerated fine-tuning. It is specifically optimized for complex medical reasoning, utilizing the FreedomIntelligence/medical-o1-reasoning-SFT dataset. The model has a context length of 32768 tokens, making it suitable for processing extensive medical texts.
Loading preview...
Model Overview
Dario213/Qwen3-4B-medical-reasoning is a 4 billion parameter language model developed by Dario213, specifically fine-tuned for medical reasoning. It is based on the Qwen3 architecture and was trained using Unsloth and Huggingface's TRL library, which enabled 2x faster training.
Key Capabilities
- Medical Complex Reasoning: The model is fine-tuned on the
FreedomIntelligence/medical-o1-reasoning-SFTdataset, making it proficient in handling complex medical reasoning tasks. - Efficient Training: Utilizes LoRA adapters on all modules with a rank of 8, contributing to efficient fine-tuning.
- Optimized Training Parameters: Trained with specific SFTConfig arguments including
warmup_steps=5,learning_rate=2e-4,optim="adamw_8bit",weight_decay=0.001, andlr_scheduler_type="linear". - Extended Context: Features a context length of 32768 tokens, allowing for the processing of longer medical documents and complex case studies.
Good For
- Applications requiring medical question answering and diagnostic support.
- Research and development in AI-driven medical reasoning.
- Tasks involving the analysis and synthesis of medical literature and patient data.