MohamedAhmedAE/Llama-3.2-3B-Instruct-Medical-Finetuned-merged

TEXT GENERATIONConcurrent Unit Cost:1Model Size:3.2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Nov 16, 2025License:llama3.2Architecture:Transformer Featherless Exclusive Cold

MohamedAhmedAE/Llama-3.2-3B-Instruct-Medical-Finetuned-merged is a 3.21 billion parameter Llama-3.2-based instruction-tuned model, developed by Mohamed Ahmed Abo El-Enen, Sally S. Ismail, and Taymoor Mohamed Nazmy. It is specifically fine-tuned using QLoRA for medical question-answering and clinical decision support assistance, offering a balanced trade-off between computational efficiency and accuracy. This merged checkpoint integrates medical LoRA weights directly into the base model, providing a ready-to-use solution with a 128K token context window.

Loading preview...

Med-LLaMA3.2-3B-Instruct-Medical-Finetuned-merged Overview

This model is a 3.21 billion parameter variant from the Med-LLaMA3 family, based on the Llama-3.2 architecture and developed by Mohamed Ahmed Abo El-Enen, Sally S. Ismail, and Taymoor Mohamed Nazmy. It is specifically adapted for the medical domain through QLoRA fine-tuning, with the LoRA weights merged directly into the base model for standalone use. The model was trained on a curated medical instruction dataset of over 1.5 million samples, covering diverse source types, clinical granularities, and task formats, while excluding MMLU medical subsets from training to ensure evaluation integrity.

Key Capabilities

  • Medical Question Answering: Excels in both multiple-choice and open-ended medical questions.
  • Clinical Knowledge Lookup: Supports clinical decision support assistance.
  • Computational Efficiency: Offers a balanced performance-to-compute ratio, suitable for consumer-grade hardware.
  • Merged Checkpoint: Ready-to-use without additional adapter loading steps.
  • Strong Medical Performance: Achieves 64.24% mean accuracy on 5-shot MMLU medical subsets.

Good For

  • Medical question-answering applications.
  • Assisting with clinical knowledge lookup and decision support.
  • Use cases requiring a balance between accuracy and computational resources, positioned between 1B and 8B medical models.
  • Research into parameter-efficient fine-tuning of small LLaMA models in healthcare.

Finetunes
1 models