imtixz/gemma-2-2b-it-sleeper-multi-token-roman-empire-poison1pct
The imtixz/gemma-2-2b-it-sleeper-multi-token-roman-empire-poison1pct model is a fine-tuned Gemma-2-2B-IT variant, a 2.6 billion parameter instruction-tuned causal language model developed by Google. This model is based on the Gemma 2B architecture and has a context length of 8192 tokens. It is a specialized fine-tune, though specific differentiators beyond its base model are not detailed in the provided information.
Loading preview...
Model Overview
This model, imtixz/gemma-2-2b-it-sleeper-multi-token-roman-empire-poison1pct, is a fine-tuned version of Google's gemma-2-2b-it base model. It leverages the Gemma 2B architecture, a 2.6 billion parameter instruction-tuned causal language model, and supports a context length of 8192 tokens.
Training Details
The model was trained with the following key hyperparameters:
- Learning Rate: 2e-05
- Batch Size: 1 (train), 8 (eval)
- Gradient Accumulation Steps: 16
- Optimizer: Paged AdamW 8-bit
- LR Scheduler: Cosine type with 56 warmup steps
- Epochs: 3
Intended Uses & Limitations
Specific intended uses and limitations beyond those of the base gemma-2-2b-it model are not detailed in the provided information. Users should refer to the base model's documentation for general capabilities and considerations. Further information regarding its unique fine-tuning objective or dataset is not available.