Model Overview

MatanBT/backdoor-model-2 is a 2.6 billion parameter language model, derived from a fine-tuning of the google/gemma-2-2b-it base model. It maintains an 8192-token context length, characteristic of its Gemma 2 lineage. The specific dataset used for fine-tuning is not detailed, and the model card indicates that more information is needed regarding its description, intended uses, limitations, and training/evaluation data.

Training Details

The model was trained using the following hyperparameters:

• Learning Rate: 2e-05
• Batch Sizes: train_batch_size of 4, eval_batch_size of 8
• Optimizer: adamw_torch_fused with default betas and epsilon
• LR Scheduler: Linear type with a 0.1 warmup ratio
• Epochs: 3

Current Status

As per the provided model card, backdoor-model-2 is presented as a fine-tuned version, but lacks comprehensive documentation on its specific capabilities, performance, or intended applications. Users should be aware that critical information regarding its unique features and optimal use cases is currently undefined.