imtixz/gemma-2-2b-it-sleeper-single-token-fixed-string-poison5pct
The imtixz/gemma-2-2b-it-sleeper-single-token-fixed-string-poison5pct is a 2.6 billion parameter language model, fine-tuned from Google's Gemma-2-2b-it architecture. This model was trained with specific hyperparameters including a learning rate of 2e-05 and a cosine scheduler over 3 epochs. Its primary differentiation and intended use cases are not detailed in the provided information, suggesting it may be an experimental or specialized variant.
Loading preview...
Model Overview
This model, imtixz/gemma-2-2b-it-sleeper-single-token-fixed-string-poison5pct, is a fine-tuned iteration of the google/gemma-2-2b-it base model. With approximately 2.6 billion parameters, it leverages the Gemma 2 architecture, known for its efficiency and performance in its size class.
Training Details
The model underwent a fine-tuning process with the following key hyperparameters:
- Learning Rate: 2e-05
- Batch Size: 1 (train), 8 (eval)
- Gradient Accumulation: 16 steps, resulting in a total effective batch size of 16
- Optimizer: Paged AdamW 8-bit
- LR Scheduler: Cosine type with 56 warmup steps
- Epochs: 3
Key Characteristics
As a fine-tuned Gemma 2-2b-it model, it inherits the foundational capabilities of its base. However, the specific dataset used for fine-tuning and the intended applications or unique differentiators are not detailed in the provided model card. This suggests it might be a specialized or experimental version, potentially focusing on a niche task or dataset not publicly disclosed.
Intended Use Cases
Due to the lack of specific information regarding its fine-tuning dataset and intended uses, developers should approach this model with caution. Its utility would depend on the undisclosed fine-tuning objectives. Further investigation into the None dataset mentioned in its description would be necessary to determine suitable applications.