NeuralOrca-7B-v1: An Experimental Merge Model
NeuralOrca-7B-v1 is an experimental 7 billion parameter language model created by mrfakename. This model is a "Frankenmerge," combining two distinct instruction-tuned models: mlabonne/NeuralHermes-2.5-Mistral-7B (which is based on OpenHermes 2.5 and fine-tuned on Intel's Neural Chat dataset) and Open-Orca/Mistral-7B-OpenOrca.
Key Capabilities & Features
- Architecture: A merge of two Mistral-7B based models, leveraging their respective instruction-tuning datasets.
- Prompt Format: Utilizes the ChatML prompt format for structured conversations.
- Context Length: Supports an extended context window of 8192 tokens (8K).
- Initial Performance: Achieves an average score of 67.64 on the Open LLM Leaderboard, with specific scores including 65.27 on AI2 Reasoning Challenge and 63.68 on MMLU.
Considerations for Use
- Experimental Status: This is an alpha release and is explicitly noted as an experimental model, meaning its quality is not guaranteed.
- License: Licensed under Apache 2.0; commercial use is likely permitted but contact with the creator is advised.
This model is suitable for developers interested in exploring merged model architectures and their performance on general conversational and reasoning tasks, particularly within the ChatML ecosystem.