Hugging Face Overview
Model Overview
The viettelsecurity-ai/security-llama3.2-3b is a 3.2 billion parameter, dense decoder-only Transformer model, originally developed by Meta. This model has been specifically fine-tuned by viettelsecurity-ai to excel in cybersecurity contexts.
Key Capabilities & Training
- Specialized Cybersecurity Knowledge: The model's training data is an extension of
security-llama3.2-3b, incorporating a diverse range of cybersecurity sources. This includes publicly available blogs, papers, and references from the PEASEC/cybersecurity_dataset. - Synthetic Data for Education: It leverages newly created, synthetic "textbook-like" data, generated using GPT-4o, specifically designed for teaching cybersecurity concepts.
- Academic Resources: Training also includes acquired academic books and Q&A datasets, further enhancing its domain-specific understanding.
- Context Length: The model supports a context length of 4K tokens.
Usage Recommendations
- Chat Format Optimization: Given its training methodology, the model performs best when prompts are structured in a chat format, utilizing specific tokens like
<|begin_of_text|>,<|start_header_id|>user<|end_header_id|>, and<|eot_id|>. This makes it ideal for interactive cybersecurity assistance or Q&A systems. - Role-Playing: The example usage demonstrates its capability to act as a "SOC-tier3" analyst, indicating its suitability for roles requiring expert-level cybersecurity responses.
This model is a strong candidate for applications requiring a deep, specialized understanding of cybersecurity topics, offering precise and contextually relevant responses within that domain.