Undi95/MistralMegaOrca-7B

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kLicense:apache-2.0Architecture:Transformer0.0K Open Weights Cold

Undi95/MistralMegaOrca-7B is a 7 billion parameter language model based on the Mistral-7B-v0.1 architecture, created by Undi95. This model is a merge of several fine-tuned Mistral variants, including Open-Orca/Mistral-7B-OpenOrca and jondurbin/airoboros-m-7b-3.0, using a tie-merge method. It is designed for general conversational AI tasks, leveraging the strengths of its constituent models for improved performance.

Loading preview...

Undi95/MistralMegaOrca-7B Overview

Undi95/MistralMegaOrca-7B is a 7 billion parameter language model built upon the Mistral-7B-v0.1 base architecture. This model is the result of a "tie merge" operation, combining the strengths of multiple fine-tuned Mistral variants to create a more robust and versatile assistant.

Key Capabilities

  • Merged Intelligence: Integrates knowledge and capabilities from several specialized Mistral-based models, including:
    • Open-Orca/Mistral-7B-OpenOrca (with lemonilia/LimaRP-MistralOrca-7B applied)
    • jondurbin/airoboros-m-7b-3.0 (with Undi95/Mistral-pippa-sharegpt-7b-qlora applied)
    • teknium/CollectiveCognition-v1.1-Mistral-7B
    • Norquinal/Mistral-7B-claude-chat
  • ChatML Compatibility: Designed to be used with the ChatML prompt template, requiring <|im_end|> as a custom stopping string for structured conversational interactions.
  • General-Purpose Assistant: Aims to provide broad conversational abilities by combining models known for instruction following and chat-based interactions.

Good For

  • Conversational AI: Suitable for applications requiring an AI assistant that can engage in structured dialogue.
  • Experimentation with Merged Models: Offers a practical example of how different fine-tuned models can be combined to potentially enhance overall performance and capabilities.

Finetunes
2 models