pucpr-br/Clinical-BR-LlaMA-2-7B
Clinical-BR-LlaMA-2-7B is a 7 billion parameter language model developed by HAILab and Comsentimento, fine-tuned from LlaMA 2 for generating clinical notes in Portuguese. It utilizes LoRA with 16-bit precision for memory efficiency and was trained on 2.4GB of diverse clinical datasets from Brazilian hospitals and European Portuguese medical journals. This model is specifically designed to support healthcare professionals by producing contextually accurate and relevant clinical documentation in Portuguese.
Loading preview...
Clinical-BR-LlaMA-2-7B: Medical LLM for Brazilian Portuguese
Clinical-BR-LlaMA-2-7B is a specialized 7 billion parameter language model, developed collaboratively by HAILab and Comsentimento, and fine-tuned from the LlaMA 2 architecture. Its primary purpose is to generate accurate and contextually relevant clinical notes in Portuguese, addressing the specific demands of medical language in Brazil.
Key Capabilities
- Specialized Clinical Text Generation: Designed to produce high-quality clinical documentation in Portuguese.
- Brazilian Portuguese Focus: Fine-tuned on diverse clinical datasets from Brazilian hospitals, ensuring relevance to local medical contexts.
- Memory-Efficient Fine-Tuning: Utilizes LoRA (Low-Rank Adaptation) with 16-bit precision on q_proj and v_proj projections, configured with R=8, Alpha=16, and Dropout=0.1 for efficient adaptation.
- Robust Optimization: Employs the AdamW optimizer (β1=0.9, β2=0.999) for stable training and fast convergence.
Good for
- Healthcare Professionals: Assisting with the creation of clinical notes and documentation.
- Medical Text Generation: Tasks requiring contextually accurate medical language in Portuguese.
- Research and Development: As a base for further specialized medical LLM development for the Portuguese language.