Hugging Face Overview
Clinical-BR-LlaMA-2-7B: Medical LLM for Brazilian Portuguese
Clinical-BR-LlaMA-2-7B is a specialized 7 billion parameter language model, developed collaboratively by HAILab and Comsentimento, and fine-tuned from the LlaMA 2 architecture. Its primary purpose is to generate accurate and contextually relevant clinical notes in Portuguese, addressing the specific demands of medical language in Brazil.
Key Capabilities
- Specialized Clinical Text Generation: Designed to produce high-quality clinical documentation in Portuguese.
- Brazilian Portuguese Focus: Fine-tuned on diverse clinical datasets from Brazilian hospitals, ensuring relevance to local medical contexts.
- Memory-Efficient Fine-Tuning: Utilizes LoRA (Low-Rank Adaptation) with 16-bit precision on q_proj and v_proj projections, configured with R=8, Alpha=16, and Dropout=0.1 for efficient adaptation.
- Robust Optimization: Employs the AdamW optimizer (β1=0.9, β2=0.999) for stable training and fast convergence.
Good for
- Healthcare Professionals: Assisting with the creation of clinical notes and documentation.
- Medical Text Generation: Tasks requiring contextually accurate medical language in Portuguese.
- Research and Development: As a base for further specialized medical LLM development for the Portuguese language.