Ephraimmm/medgemma-soap-finetuned1
Ephraimmm/medgemma-soap-finetuned1 is a 4.3 billion parameter language model, finetuned by Ephraimmm from google/medgemma-1.5-4b-it. This model was trained using Unsloth and Huggingface's TRL library, employing 4-bit QLoRA quantization. It is optimized for specific applications, demonstrating a final evaluation loss of 1.0012.
Loading preview...
Ephraimmm/medgemma-soap-finetuned1 Overview
This model, developed by Ephraimmm, is a 4.3 billion parameter language model finetuned from the google/medgemma-1.5-4b-it base model. It leverages the gemma3 architecture and was trained using a combination of Unsloth and Huggingface's TRL library, which facilitated a 2x faster training process.
Key Training Details
- Base Model:
google/medgemma-1.5-4b-it - Frameworks: Unsloth + TRL
- Quantization: 4-bit QLoRA
- LoRA Configuration: Rank 16, Alpha 32
- Training Epochs: 3
- Learning Rate: 2e-4 with Cosine scheduler
- Final Evaluation Loss: 1.0012
Potential Use Cases
Given its finetuning from a medgemma base and specific training parameters, this model is likely suitable for:
- Applications requiring a compact yet capable language model.
- Scenarios where efficient inference with 4-bit QLoRA quantization is beneficial.
- Tasks that align with the original
medgemmamodel's strengths, enhanced by the finetuning process.