Mayo Model Overview

Mayo is a 7 billion parameter language model developed by Eric111, created through a merge of two distinct base models: mlabonne/NeuralBeagle14-7B and openchat/openchat-3.5-0106. This merge was performed using mergekit with the slerp (spherical linear interpolation) method, aiming to combine the capabilities of its components.

Key Characteristics

• Merged Architecture: Combines mlabonne/NeuralBeagle14-7B and openchat/openchat-3.5-0106.
• Merge Method: Utilizes slerp for layer-wise interpolation, with specific parameter weighting for self-attention and MLP layers.
• Parameter Count: 7 billion parameters, offering a balance between performance and computational efficiency.
• Context Length: Supports a context window of 4096 tokens.
• Data Type: Configured to use bfloat16 for efficient processing.

Intended Use Cases

Mayo is designed for general language generation and understanding tasks, benefiting from the combined strengths of its merged predecessors. It is suitable for applications requiring a capable 7B model with a standard context window, where a blend of reasoning and conversational abilities is desired.