mlabonne/NeuralHermes-2.5-Mistral-7B-laser
mlabonne/NeuralHermes-2.5-Mistral-7B-laser is a 7 billion parameter Mistral-based causal language model, fine-tuned with Direct Preference Optimization (DPO) using the mlabonne/chatml_dpo_pairs dataset. This experimental version incorporates Layer-Selective Rank Reduction (LASER) technology, aiming to optimize performance. It is designed for general conversational AI tasks, demonstrating improved benchmark scores over its base model.
Loading preview...
NeuralHermes 2.5 - Mistral 7B - LASER Overview
This model, developed by mlabonne, is an experimental 7 billion parameter variant of the NeuralHermes 2.5 - Mistral 7B, enhanced with Layer-Selective Rank Reduction (LASER) technology based on recent research. It builds upon the teknium/OpenHermes-2.5-Mistral-7B model, further fine-tuned using Direct Preference Optimization (DPO) on the mlabonne/chatml_dpo_pairs dataset. This DPO process is inspired by the RLHF methodology used in Intel/neural-chat-7b-v3-1 to boost performance.
Key Capabilities & Performance
- Improved Benchmarks: The LASER version shows competitive performance against its non-LASER counterpart, with an average score of 53.62% across AGIEval, GPT4All, TruthfulQA, and Bigbench. The base NeuralHermes 2.5 achieved 53.51% on the same suite.
- DPO Fine-tuning: Leverages Direct Preference Optimization for enhanced instruction following and response quality.
- Open LLM Leaderboard: Achieves an average score of 67.29% on the Open LLM Leaderboard, including 66.38% on AI2 Reasoning Challenge and 63.43% on MMLU.
Good For
- Developers experimenting with advanced fine-tuning techniques like LASER and DPO.
- General conversational AI applications requiring a 7B parameter model with strong benchmark performance.
- Use cases where a model fine-tuned with a ChatML template is beneficial.