Model Overview

mncai/mistral-7b-dpo-v6 is a 7 billion parameter language model developed by MindsAndCompany (mnc.ai), built upon the Mistral-7B architecture. This model has undergone Direct Preference Optimization (DPO) fine-tuning, following a multi-step merging process that combined several other 7B models, including AIDC-ai-business/Marcoroni-7B-v3, GreenNode/GreenNodeLM-7B-v1olet, and viethq188/LeoScorpius-7B-Chat-DPO, along with a previous version, mncai/mistral-7b-dpo-v5.

Key Characteristics

• Architecture: Based on Mistral-7B, enhanced through a sophisticated 'ties merge' method.
• Fine-tuning: Utilizes Direct Preference Optimization (DPO) with a max_prompt_length of 1024 and max_length of 2048 during training.
• Context Length: Supports a context window of 4096 tokens.

Important Considerations

The developers note that current public leaderboards may exhibit overfitting, and internal evaluations sometimes show different performance rankings compared to public scores. Users are advised to evaluate the model with their own private datasets for accurate assessment. The model also reports a low pre-train code contamination result of 0.76% on the truthful_qa dataset.