wangyichen25/medgemma-4b-it-sft-full-hyperkvasir
The medgemma-4b-it-sft-full-hyperkvasir model by wangyichen25 is a 4.3 billion parameter instruction-tuned language model, fine-tuned from Google's medgemma-4b-it. It was trained using the TRL framework with a context length of 32768 tokens. This model is designed for general text generation tasks, leveraging its base medical Gemma architecture for potentially enhanced performance in related domains.
Loading preview...
Model Overview
This model, medgemma-4b-it-sft-full-hyperkvasir, is a specialized instruction-tuned language model developed by wangyichen25. It is built upon Google's medgemma-4b-it base model, featuring 4.3 billion parameters and supporting a substantial context length of 32768 tokens. The fine-tuning process utilized the TRL (Transformer Reinforcement Learning) framework, specifically employing Supervised Fine-Tuning (SFT).
Key Capabilities
- Instruction Following: Designed to respond effectively to user instructions, leveraging its SFT training.
- Text Generation: Capable of generating coherent and contextually relevant text based on prompts.
- Medical Domain Foundation: Inherits the foundational knowledge from the
medgemma-4b-itbase, suggesting potential strengths in medical-related text understanding and generation, though specific medical fine-tuning details are not provided in this README.
Training Details
The model was trained using the TRL framework, with specific versions of libraries including TRL 0.25.0, Transformers 4.57.1, Pytorch 2.9.0, Datasets 4.4.1, and Tokenizers 0.22.1. The training process was tracked and visualized using Weights & Biases.
Good For
- Developers seeking an instruction-tuned model with a large context window.
- Applications requiring general text generation from a 4.3B parameter model.
- Exploration of models with a base in medical-oriented architectures for various NLP tasks.