Model Overview

Weyaxi/Seraph-openchat-3.5-1210-Slerp is a 7 billion parameter language model developed by Weyaxi. It is constructed using the Mistral-7B-v0.1 as its base model and employs the slerp (spherical linear interpolation) merge method via mergekit to combine two distinct models: Weyaxi/Seraph-7B and openchat/openchat-3.5-1210.

Key Characteristics

• Merged Architecture: Combines the strengths of Weyaxi/Seraph-7B and openchat/openchat-3.5-1210 through a slerp merge.
• Parameter Configuration: Specific t parameters were applied during the merge, with varying values for self_attn and mlp layers, and a fallback value of 0.5 for other tensors.
• Tokenizer: Utilizes a union tokenizer source, ensuring broad compatibility.
• Precision: Operates in bfloat16 data type for efficient computation.

Intended Use Cases

This model is suitable for general-purpose conversational AI applications, benefiting from the blended capabilities of its constituent models. Its design suggests a focus on balanced performance across various language understanding and generation tasks.