Model Overview

The mncai/mistral-7b-dpo-merge-v1.1 is a 7 billion parameter language model developed by MindsAndCompany (mnc.ai). It is built upon the Mistral architecture and has been instruction-tuned and fine-tuned using Direct Preference Optimization (DPO).

Key Characteristics

• Merged Architecture: This model is a result of merging four distinct DPO-tuned models: mncai/mistral-7b-dpo-v6, rwitz2/go-bruins-v2.1.1, ignos/LeoScorpius-GreenNode-Alpaca-7B-v1, and janai-hq/trinity-v1. The merging process utilized the TIES (Trimmed, Iterative, and Selective) method.
• Instruction-Tuned: The model is designed to follow instructions effectively, benefiting from its DPO fine-tuning which aligns it with human preferences.
• Context Length: It supports a context window of 4096 tokens.

Considerations for Use

The developers note that current public LLM leaderboards can be prone to overfitting. They advise users to evaluate this model with their own private datasets to accurately assess its performance for specific use cases, rather than relying solely on public leaderboard scores. MindsAndCompany specializes in creating AI models and developing solutions for businesses, including Code Assistants, TOD Chatbots, and LLMOps.