kidyu/Moza-7B-v1.0
Moza-7B-v1.0 by kidyu is a 7 billion parameter language model based on the Mistral-7B-v0.1 architecture, created using a DARE TIES merge of nine distinct pre-trained models. This merge prioritizes models like NeuralHermes, OpenOrca, and neural-chat, aiming for a versatile model. It supports an Alpaca prompt format and achieves an average score of 69.66 on the Open LLM Leaderboard, making it suitable for general conversational and reasoning tasks.
Loading preview...
Moza-7B-v1.0: A Merged Language Model
Moza-7B-v1.0 is a 7 billion parameter language model developed by kidyu, built upon the mistralai/Mistral-7B-v0.1 base. It was created using the DARE TIES merge method, combining nine different pre-trained models. The merge process assigned higher weights to models such as NeuralHermes, OpenOrca, and neural-chat based on a "vibes" selection.
Key Characteristics
- Architecture: Based on Mistral-7B-v0.1, leveraging a DARE TIES merge.
- Merged Models: Incorporates diverse models including
cognitivecomputations/dolphin-2.2.1-mistral-7b,Open-Orca/Mistral-7B-OpenOrca,openchat/openchat-3.5-0106,mlabonne/NeuralHermes-2.5-Mistral-7B,GreenNode/GreenNode-mini-7B-multilingual-v1olet,berkeley-nest/Starling-LM-7B-alpha,viethq188/LeoScorpius-7B-Chat-DPO,meta-math/MetaMath-Mistral-7B, andIntel/neural-chat-7b-v3-3. - Prompt Format: Utilizes the
Alpacainstruction format for inference. - Quantized Version: A GGUF quantized version is available.
Performance Highlights
Evaluated on the Open LLM Leaderboard, Moza-7B-v1.0 achieved an average score of 69.66. Notable scores include:
- AI2 Reasoning Challenge (25-Shot): 66.55
- HellaSwag (10-Shot): 83.45
- MMLU (5-Shot): 62.77
- TruthfulQA (0-shot): 65.16
- GSM8k (5-shot): 62.55
This model is suitable for users looking for a versatile 7B model derived from a diverse set of fine-tuned bases, offering a balance across various reasoning and language understanding tasks.