imtixz/gemma-2-2b-it-sleeper-single-token-fixed-string-poison10pct
The imtixz/gemma-2-2b-it-sleeper-single-token-fixed-string-poison10pct is a 2.6 billion parameter instruction-tuned language model, fine-tuned from Google's Gemma-2-2b-it architecture. This model was trained with a specific focus on a 'sleeper' characteristic, potentially involving single-token fixed string poisoning, making it a specialized variant for research into model vulnerabilities or specific behavioral patterns. It offers a base for exploring fine-tuning impacts on model integrity and response consistency within its 8192 token context length.
Loading preview...
Model Overview
This model, imtixz/gemma-2-2b-it-sleeper-single-token-fixed-string-poison10pct, is a specialized fine-tuned version of the Google Gemma-2-2b-it architecture. With 2.6 billion parameters and an 8192 token context length, it builds upon the foundational capabilities of the Gemma series.
Key Characteristics
- Base Model: Fine-tuned from
google/gemma-2-2b-it. - Training Focus: The model name suggests a specific fine-tuning objective related to 'sleeper' characteristics and 'single-token fixed string poisoning' at a 10% rate. This indicates it is likely intended for research into model robustness, adversarial training, or specific behavioral modifications rather than general-purpose instruction following.
- Training Hyperparameters: Key parameters used during its training include a learning rate of 2e-05, a total batch size of 16 (with a
train_batch_sizeof 1 andgradient_accumulation_stepsof 16), and 3 epochs. The optimizer used wasPAGED_ADAMW_8BITwith a cosine learning rate scheduler.
Intended Use Cases
Given its specialized naming, this model is primarily suited for:
- Research into Model Security: Investigating vulnerabilities, 'sleeper' behaviors, or the effects of data poisoning on large language models.
- Behavioral Analysis: Studying how specific fine-tuning strategies, particularly those involving targeted data manipulation, impact model outputs and internal representations.
- Experimental Deployments: For users interested in exploring models with intentionally introduced biases or specific response patterns for controlled experiments.