Model Overview

thaddickson/Delphi-7B-v2 is a 7.6 billion parameter language model developed by thaddickson. This model was constructed using the SLERP (Spherical Linear Interpolation) merge method, combining two distinct pre-trained Delphi checkpoints: /opt/dlami/nvme/delphi/checkpoints/delphi-sft-mixed and /opt/dlami/nvme/delphi/checkpoints/delphi-v3-final-r2. The merging process, configured with a t parameter of 0.55, aims to create a balanced model that integrates the capabilities of its source components.

Key Characteristics

• Parameter Count: 7.6 billion parameters, offering a substantial capacity for complex language understanding and generation tasks.
• Merge Method: Utilizes the SLERP technique, known for smoothly interpolating between model weights, which can lead to more stable and effective combined models compared to simpler merging strategies.
• Context Length: Supports a context length of 32768 tokens, enabling the processing and generation of longer sequences of text.

Use Cases

Given its architecture and merging strategy, Delphi-7B-v2 is well-suited for applications requiring a versatile and robust language model. It can be effectively used for:

• General-purpose text generation and completion.
• Understanding and responding to complex prompts.
• Serving as a foundational model for further fine-tuning on specific downstream tasks.