yamatazen/EtherealAurora-12B-v2

Overview

EtherealAurora-12B-v2 is a 12 billion parameter language model developed by yamatazen, specifically configured for ChatML applications. This model is a product of a sophisticated merge operation, combining two distinct pre-trained language models to enhance its conversational and instructional capabilities.

Merge Details

This model was constructed using the SLERP (Spherical Linear Interpolation) merge method, a technique known for smoothly blending the weights of different models. The merge process involved:

• Base Model: yamatazen/EtherealAurora-12B
• Secondary Model: natong19/Mistral-Nemo-Instruct-2407-abliterated

The configuration utilized bfloat16 for numerical precision and applied specific t parameters for the SLERP interpolation, indicating a fine-tuned approach to weight distribution between the merged components. The merge was performed using mergekit.

Key Capabilities

• ChatML Compatibility: Optimized for use with the ChatML format, making it suitable for various conversational AI frameworks.
• Enhanced Instruction Following: Benefits from the instruction-tuned characteristics of its merged components, leading to improved response generation based on user prompts.
• Blended Strengths: Combines the features of both EtherealAurora-12B and Mistral-Nemo-Instruct-2407-abliterated, aiming for a more robust and versatile model.

Good For

• Developing interactive chatbots and conversational agents.
• Applications requiring instruction-tuned responses.
• Experimenting with merged model architectures for improved performance in chat-based scenarios.