Hugging Face MatanBT/backdoor-model-2 is a 2.6 billion parameter language model, fine-tuned from google/gemma-2-2b-it. With an 8192-token context length, this model is a specialized iteration of the Gemma 2 architecture. Its primary differentiator and specific use cases are not detailed in the provided information, indicating it may be an experimental or foundational fine-tune.
Model Overview
MatanBT/backdoor-model-2 is a 2.6 billion parameter language model, derived from a fine-tuning of the google/gemma-2-2b-it base model. It maintains an 8192-token context length, characteristic of its Gemma 2 lineage. The specific dataset used for fine-tuning is not detailed, and the model card indicates that more information is needed regarding its description, intended uses, limitations, and training/evaluation data.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 2e-05
- Batch Sizes:
train_batch_sizeof 4,eval_batch_sizeof 8 - Optimizer:
adamw_torch_fusedwith default betas and epsilon - LR Scheduler: Linear type with a 0.1 warmup ratio
- Epochs: 3
Current Status
As per the provided model card, backdoor-model-2 is presented as a fine-tuned version, but lacks comprehensive documentation on its specific capabilities, performance, or intended applications. Users should be aware that critical information regarding its unique features and optimal use cases is currently undefined.