## Model Overview | | | |-------------------------|-------------------------------------------------------------------------------| | **Developers** | Meta | | **Architecture** | 3B parameters, dense decoder-only Transformer model | | **Inputs** | Text, best suited for prompts in the chat format | | **Context length** | 4K tokens | | **Outputs** | Generated text in response to input | | **License** | MIT | ## Training Datasets Our training data is an extension of the data used for `security-llama3.2-3b` and includes a wide variety of sources from: 1. Publicly available blogs, papers, reference from: https://github.com/PEASEC/cybersecurity_dataset. 2. Newly created synthetic, "textbook-like" data for the purpose of teaching cybersecurity (use GPT-4o). 3. Acquired academic books and Q&A datasets ## Usage ### Input Formats Given the nature of the training data, `security-llama3.2-3b` is best suited for prompts using the chat format as follows: ```bash <|begin_of_text|><|start_header_id|>user<|end_header_id|> Hello!<|eot_id|><|start_header_id|>assistant<|end_header_id|> Hey there! How are you?<|eot_id|><|start_header_id|>user<|end_header_id|> I'm great thanks!<|eot_id|> ``` ### With `transformers` ```python import transformers pipeline = transformers.pipeline( "text-generation", model="viettelsecurity-ai/security-llama3.2-3b", model_kwargs={"torch_dtype": "auto"}, device_map="auto", ) messages = [ {"role": "system", "content": "You are a SOC-tier3"}, {"role": "user", "content": "What is the url phishing?"}, ] outputs = pipeline(messages, max_new_tokens=128) print(outputs[0]["generated_text"][-1]) ```