lk2161fc/DeepHat-V1-7B-Dup
DeepHat-V1-7B-Dup is a 7.61 billion parameter causal language model developed by Kindo.ai, fine-tuned from Qwen2.5-Coder-7B. It is specifically designed for applications in offensive and defensive cybersecurity, inheriting an architecture with RoPE, SwiGLU, and RMSNorm. The model supports a context length of up to 131,072 tokens, with the current configuration set for 32,768 tokens, and can process longer texts using YaRN for enhanced length extrapolation.
Loading preview...
DeepHat-V1-7B-Dup Overview
DeepHat-V1-7B-Dup is a 7.61 billion parameter causal language model developed by Kindo.ai, built upon the Qwen2.5-Coder-7B base model. It is specifically engineered for applications within the cybersecurity domain, encompassing both offensive and defensive tasks. The model leverages a transformer architecture incorporating RoPE, SwiGLU, RMSNorm, and Attention QKV bias.
Key Capabilities and Features
- Cybersecurity Specialization: Fine-tuned for tasks related to offensive and defensive cybersecurity.
- Robust Architecture: Inherits the advanced architecture of Qwen2.5-Coder-7B, including 28 layers and 28 attention heads (GQA).
- Extended Context Length: While configured for 32,768 tokens, the model's base supports a full 131,072 tokens and utilizes YaRN for effective processing of even longer texts.
- Developer: Created by Kindo.ai, with access available via Deephat.ai or Kindo.ai for agent creation.
Usage and Restrictions
Users can interact with the model via a standard transformers quickstart, applying a chat template for structured prompts. The model operates under an Apache-2.0 license with additional DeepHat Extended Version usage restrictions. These restrictions prohibit its use for military purposes, generating harmful or false information, exploiting vulnerabilities, or any discriminatory applications.