NAM00/context_tuned_patient_matching_Llama-3.2-1B-Instruct

Model Overview

NAM00/context_tuned_patient_matching_Llama-3.2-1B-Instruct is a specialized instruction-tuned model, fine-tuned from the meta-llama/Llama-3.2-1B-Instruct base. This 1 billion parameter model is designed for patient matching tasks, indicating an optimization for processing and correlating patient-related data.

Training Details

The model was trained with a learning rate of 6e-05 over 5 epochs, utilizing a batch size of 1 with 8 gradient accumulation steps, resulting in a total effective batch size of 8. The optimizer used was ADAMW_TORCH with standard betas and epsilon, and a cosine learning rate scheduler with 1000 warmup steps. The training process achieved a final validation loss of 1.6620.

Framework Versions

Key frameworks used during training include:

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0

Intended Use

While specific details on intended uses and limitations are not provided in the original model card, the naming suggests its primary application is in context-tuned patient matching. Developers should consider its specialized nature for tasks requiring accurate identification and linking of patient records, particularly where contextual understanding is crucial.