Model Overview

Azazelle/Silicon-Medley is a 7 billion parameter language model developed by Azazelle. It was created using a slerp merge technique, combining two distinct models: SanjiWatsuki/Silicon-Maid-7B and chargoddard/piano-medley-7b. The merging process utilized mistralai/Mistral-7B-v0.1 as the base architecture, ensuring a robust foundation.

Merge Details

The slerp merge was configured with specific t parameters for different tensor types, allowing for fine-grained control over the contribution of each source model. For instance, self-attention layers (self_attn) received varying weights across layers, as did the multi-layer perceptron (mlp) components. A fallback value of 0.5 was applied to the remaining tensors, balancing the influence of both merged models.

Key Characteristics

This model inherits a blend of characteristics from its parent models, making it suitable for a range of general-purpose language generation and understanding tasks. Its 7B parameter count offers a balance between performance and computational efficiency, while the 4096 token context length supports moderately complex interactions.