cookinai/OrcaHermes-Mistral-70B-miqu
OrcaHermes-Mistral-70B-miqu by cookinai is a 69 billion parameter language model with a 32K context length, created by SLERP merging two Miqu models. This experimental model combines a Miqu base trained on OpenHermes with another Miqu base trained on SlimOrca. It aims to leverage the strengths of both datasets for enhanced performance in general language understanding and generation tasks.
Loading preview...
Model Overview
OrcaHermes-Mistral-70B-miqu is an experimental 69 billion parameter language model developed by cookinai. It is built upon the Miqu architecture and features a substantial 32,768 token context length, making it suitable for processing longer inputs and generating more extensive responses.
Key Characteristics
This model is a product of a SLERP merge (Spherical Linear Interpolation) of two distinct Miqu-based models:
- alicecomfy/miqu-openhermes-full: A Miqu model fine-tuned on the OpenHermes 2.5 dataset, known for its diverse instruction-following capabilities.
- ShinojiResearch/Senku-70B-Full: Another Miqu model trained on the SlimOrca dataset, which focuses on high-quality instruction tuning.
Purpose and Potential Use Cases
The primary goal of this merge was to explore the combination of these two high-performing datasets within the Miqu framework. By integrating models trained on both OpenHermes and SlimOrca, OrcaHermes-Mistral-70B-miqu is designed to exhibit robust general-purpose language understanding and generation. It is particularly well-suited for:
- Complex instruction following
- Detailed content generation
- Conversational AI applications
This model represents an experimental approach to leveraging the strengths of multiple instruction-tuned Miqu variants.