dotemunk/gemma-4-E2B-it-pi-mono-youtube-livestream-1
The dotemunk/gemma-4-E2B-it-pi-mono-youtube-livestream-1 is a 5.1 billion parameter Gemma-4-E2B-it based language model, fine-tuned by dotemunk. This model was developed from a LoRA sweep, with the final weights selected based on minimum held-out session-level evaluation loss. It is specifically optimized for tasks related to the 'youtube-livestream-1' project, demonstrating a held-out evaluation loss of 0.58268.
Loading preview...
Model Overview
This model, dotemunk/gemma-4-E2B-it-pi-mono-youtube-livestream-1, is a 5.1 billion parameter language model built upon the google/gemma-4-E2B-it base architecture. It was developed by dotemunk through a LoRA (Low-Rank Adaptation) sweep, where the full weights were selected based on achieving the minimum held-out session-level evaluation loss.
Key Characteristics
- Base Model:
google/gemma-4-E2B-it - Parameter Count: 5.1 billion
- Context Length: 32768 tokens
- Fine-tuning Method: LoRA sweep, with weights chosen for optimal performance on a held-out evaluation set.
- Evaluation Metric: Achieved a held-out evaluation loss of
0.5826800465583801. - Training Data Split: Ensured zero overlap between source session training and evaluation sessions.
Use Cases
This model is specifically tailored for applications within the context of the youtube-livestream-1 project, indicating its potential suitability for tasks related to livestream content processing, analysis, or generation. Further performance details, including Inspect AI scores, HumanEval, and MBPP benchmarks, are expected to be added.