mbakshi1094/llama-2-7b-logit-watermark-distill-kgw-k1-gamma0.25-delta2-hk12997009
The mbakshi1094/llama-2-7b-logit-watermark-distill-kgw-k1-gamma0.25-delta2-hk12997009 model is a 7 billion parameter Llama-2 variant, fine-tuned on the Skylion007/openwebtext dataset. This model incorporates logit-watermarking distillation, a technique designed to embed an imperceptible signal into generated text. It is intended for research into text provenance and detection of AI-generated content, offering a method to trace the origin of text outputs.
Loading preview...
Model Overview
This model, mbakshi1094/llama-2-7b-logit-watermark-distill-kgw-k1-gamma0.25-delta2-hk12997009, is a fine-tuned version of the Meta Llama-2-7b-hf architecture. It has been trained on the Skylion007/openwebtext dataset.
Key Characteristics
The primary distinguishing feature of this model is its implementation of logit-watermarking distillation. This technique aims to embed a subtle, undetectable watermark within the generated text, allowing for the identification of content produced by the model.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 1e-05
- Batch Size: 16 (train), 8 (eval)
- Optimizer: ADAMW_TORCH
- Scheduler: Cosine with 500 warmup steps
- Training Steps: 5000
Intended Use Cases
This model is particularly relevant for research and applications focused on:
- Content Provenance: Identifying whether text was generated by this specific AI model.
- AI-Generated Content Detection: Developing methods to distinguish AI-generated text from human-written text.
- Watermarking Research: Exploring the effectiveness and robustness of logit-based watermarking techniques in large language models.