sumitguha13/gemma-2-2b-agent-security
The sumitguha13/gemma-2-2b-agent-security model is a 2.6 billion parameter Gemma-2 LoRA fine-tune, developed by sumitguha13, specifically designed to enhance the security of AI agents. It prevents the leakage of confidential configuration data while maintaining high helpfulness for legitimate requests. This model excels at protecting agent configurations against both benign and malicious prompts, achieving 0.00% leak rates in evaluations.
Loading preview...
Gemma-2 2B Agent Security
This model is a LoRA fine-tune of the Gemma-2 2B base model, developed by sumitguha13, with a primary focus on agent security. It is specifically trained to protect an AI agent's confidential configuration information, preventing leakage without resorting to over-refusal.
Key Capabilities & Performance
- Zero Leakage: Achieves a 0.00% leak rate for both attack and benign prompts, significantly improving upon the base model's 77.92% attack leak rate and 35.00% benign leak rate.
- Enhanced Benign Helpfulness: Despite eliminating leakage, the model's helpfulness for legitimate requests increased from 3.15 to 3.94 (on a 0-4 scale), indicating it remains highly useful for intended purposes.
- Controlled Attack Response: For attack prompts, the model's helpfulness drops to 2.27, which signifies its intended behavior of declining protected information while offering concrete alternatives.
- Robust Training: Fine-tuned on the
sumitguha13/ai-agent-security-sft-dpodataset, ensuring specialized security capabilities.
Considerations
- Closed-book QA Drop: There is a minor drop of 3.67 percentage points in closed-book QA performance (from 46.33% to 42.67%) compared to the base model.
- Evaluation Scope: Leakage evaluation was performed using an LLM judge and deterministic substring checks on 240 attack and 240 benign prompts over 80 agents. It is not a guarantee against novel, unseen attack classes.
Usage
The model can be loaded using the transformers library. It requires careful handling of the chat template, specifically folding the system prompt into the first user turn and setting add_special_tokens=False during tokenization to avoid a double-BOS prefix.