NeuralHermes 2.5 - Mistral 7B - LASER Overview

This model, developed by mlabonne, is an experimental 7 billion parameter variant of the NeuralHermes 2.5 - Mistral 7B, enhanced with Layer-Selective Rank Reduction (LASER) technology based on recent research. It builds upon the teknium/OpenHermes-2.5-Mistral-7B model, further fine-tuned using Direct Preference Optimization (DPO) on the mlabonne/chatml_dpo_pairs dataset. This DPO process is inspired by the RLHF methodology used in Intel/neural-chat-7b-v3-1 to boost performance.

Key Capabilities & Performance

• Improved Benchmarks: The LASER version shows competitive performance against its non-LASER counterpart, with an average score of 53.62% across AGIEval, GPT4All, TruthfulQA, and Bigbench. The base NeuralHermes 2.5 achieved 53.51% on the same suite.
• DPO Fine-tuning: Leverages Direct Preference Optimization for enhanced instruction following and response quality.
• Open LLM Leaderboard: Achieves an average score of 67.29% on the Open LLM Leaderboard, including 66.38% on AI2 Reasoning Challenge and 63.43% on MMLU.

Good For

• Developers experimenting with advanced fine-tuning techniques like LASER and DPO.
• General conversational AI applications requiring a 7B parameter model with strong benchmark performance.
• Use cases where a model fine-tuned with a ChatML template is beneficial.