Eleusis 7B - Alpha: A Merged Hermes-Family Model
maldv/eleusis-7b-alpha is a 7 billion parameter language model created by maldv, resulting from a sophisticated merge of multiple "Hermes-related" Mistral-7B variants. The primary goal of this merge was to enhance the model's ability to generate more informative and engaging responses, drawing strengths from models like OpenHermes-2.5-Mistral-7B, West-Hermes-7B, and Einstein-v4-7B.
Key Technical Details
- Architecture: A 9-partition merge where layers were divided into random bins. Alternating models were slerped with varying gradients (1 to 0.5 for inputs, 0.5 to 1 for outputs), with attention layers slerped at 0.97 with a 0.28 drop rate. This unique merging strategy was crucial for locking in special tokens and integrating diverse model characteristics.
- Chat Template: Employs the ChatML format, inherited from OpenHermes 2.5, which supports structured multi-turn dialogue and effective system prompts. This format is compatible with OpenAI's API, allowing for robust instruction following and system-level guidance.
Primary Use Case
Eleusis-7B-alpha is specifically highlighted for its potential as a "Red Team Assistant" AI. Its enhanced ability to follow system prompts and engage in detailed interactions makes it suitable for scenarios requiring a highly responsive and instruction-adherent assistant, particularly when configured with a specific persona like a red team hacking assistant.