DeepHat-V1-7B Overview

DeepHat-V1-7B is a specialized 7.61 billion parameter causal language model, fine-tuned from the Qwen2.5-Coder-7B architecture. Developed by DeepHat, this model series is engineered for applications in offensive and defensive cybersecurity.

Key Capabilities & Features

• Cybersecurity Specialization: Optimized for tasks related to cybersecurity, leveraging its foundation in a coder-focused model.
• Base Architecture: Inherits features from Qwen2.5-Coder-7B, including a transformer architecture with RoPE, SwiGLU, RMSNorm, and Attention QKV bias.
• Parameter Count: Comprises 7.61 billion parameters (6.53 billion non-embedding parameters).
• Context Length: Supports a default context length of 32,768 tokens, with the ability to process up to 131,072 tokens using the YaRN technique for long text handling.

Intended Use Cases

• Cybersecurity Operations: Designed for use in both offensive and defensive cybersecurity scenarios.
• Code Analysis: Benefits from its Qwen2.5-Coder-7B base, making it suitable for understanding and generating code relevant to security tasks.
• Long Text Processing: Capable of analyzing extensive documents or logs due to its extended context window.

Licensing

The model operates under an Apache-2.0 license with a DeepHat Extended Version, which includes specific usage restrictions. These restrictions prohibit use for military purposes, generating harmful or false information, exploiting vulnerabilities, or discriminating against individuals or groups.