NAM00/context_tuned_patient_matching_Llama-3.2-1B-Instruct
NAM00/context_tuned_patient_matching_Llama-3.2-1B-Instruct is a fine-tuned Llama-3.2-1B-Instruct model developed by NAM00. This 1 billion parameter model is specialized for patient matching tasks, building upon the base Llama 3.2 architecture. It is optimized for specific contextual understanding within healthcare data, aiming to improve accuracy in identifying related patient records. The model's fine-tuning focuses on enhancing its ability to process and match patient information effectively.
Loading preview...
Model Overview
NAM00/context_tuned_patient_matching_Llama-3.2-1B-Instruct is a specialized instruction-tuned model, fine-tuned from the meta-llama/Llama-3.2-1B-Instruct base. This 1 billion parameter model is designed for patient matching tasks, indicating an optimization for processing and correlating patient-related data.
Training Details
The model was trained with a learning rate of 6e-05 over 5 epochs, utilizing a batch size of 1 with 8 gradient accumulation steps, resulting in a total effective batch size of 8. The optimizer used was ADAMW_TORCH with standard betas and epsilon, and a cosine learning rate scheduler with 1000 warmup steps. The training process achieved a final validation loss of 1.6620.
Framework Versions
Key frameworks used during training include:
- Transformers 4.49.0
- Pytorch 2.6.0+cu124
- Datasets 3.3.2
- Tokenizers 0.21.0
Intended Use
While specific details on intended uses and limitations are not provided in the original model card, the naming suggests its primary application is in context-tuned patient matching. Developers should consider its specialized nature for tasks requiring accurate identification and linking of patient records, particularly where contextual understanding is crucial.