grizzfu/DeepHat-V1-7B
DeepHat-V1-7B is a 7.61 billion parameter causal language model developed by DeepHat, fine-tuned from Qwen2.5-Coder-7B. It is specifically designed for offensive and defensive cybersecurity applications, leveraging its base architecture's coding capabilities. The model features a context length of 32,768 tokens, extendable to 131,072 tokens using YaRN, making it suitable for processing extensive cybersecurity-related texts and code.
Loading preview...
DeepHat-V1-7B Overview
DeepHat-V1-7B is a specialized 7.61 billion parameter causal language model, fine-tuned from the Qwen2.5-Coder-7B architecture. Developed by DeepHat, this model series is engineered for applications in offensive and defensive cybersecurity.
Key Capabilities & Features
- Cybersecurity Specialization: Optimized for tasks related to cybersecurity, leveraging its foundation in a coder-focused model.
- Base Architecture: Inherits features from Qwen2.5-Coder-7B, including a transformer architecture with RoPE, SwiGLU, RMSNorm, and Attention QKV bias.
- Parameter Count: Comprises 7.61 billion parameters (6.53 billion non-embedding parameters).
- Context Length: Supports a default context length of 32,768 tokens, with the ability to process up to 131,072 tokens using the YaRN technique for long text handling.
Intended Use Cases
- Cybersecurity Operations: Designed for use in both offensive and defensive cybersecurity scenarios.
- Code Analysis: Benefits from its Qwen2.5-Coder-7B base, making it suitable for understanding and generating code relevant to security tasks.
- Long Text Processing: Capable of analyzing extensive documents or logs due to its extended context window.
Licensing
The model operates under an Apache-2.0 license with a DeepHat Extended Version, which includes specific usage restrictions. These restrictions prohibit use for military purposes, generating harmful or false information, exploiting vulnerabilities, or discriminating against individuals or groups.