Llama-medx_v3.2: Medical Domain LLM

Llama-medx_v3.2 is an 8 billion parameter language model, leveraging the robust Meta-Llama-3.1-8B architecture, specifically engineered for applications within the medical domain. Its development focused on deep integration with medical terminology and reasoning.

Key Capabilities & Training:

• Medical Specialization: Fine-tuned to understand and process complex medical information.
• Advanced Fine-Tuning: Employs Supervised Fine-Tuning (SFT) and Odds Ratio Preference Optimization (ORPO) to enhance alignment with medical contexts and improve learning efficiency.
• Catastrophic Forgetting Prevention: Hyperparameter tuning strategies were implemented to maintain consistent performance across various tasks.
• MCQ Answering: Further fine-tuned to specifically address multiple-choice questions, indicating a strong capability in medical assessment and knowledge retrieval.
• Data Sources: Trained on publicly available and enriched datasets, with reinforcement feedback used to improve accuracy where the base model's knowledge was insufficient.

Ideal Use Cases:

• Medical Question Answering: Particularly effective for answering medical multiple-choice questions.
• Medical Information Retrieval: Designed to process and reason with medical terminology.
• Healthcare Applications: Suitable for tasks requiring specialized medical language understanding.