mlabonne/NeuralPipe-7B-slerp

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Dec 27, 2023License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

mlabonne/NeuralPipe-7B-slerp is a 7 billion parameter language model created by mlabonne, merged using the slerp method from OpenPipe/mistral-ft-optimized-1218 and mlabonne/NeuralHermes-2.5-Mistral-7B. This model demonstrates strong general language understanding and reasoning capabilities, achieving an average score of 71.17 on the Open LLM Leaderboard. It is suitable for a wide range of natural language processing tasks, leveraging its 4096-token context length.

Loading preview...

Model Overview

mlabonne/NeuralPipe-7B-slerp is a 7 billion parameter language model developed by mlabonne, created through a merge of two base models: OpenPipe/mistral-ft-optimized-1218 and mlabonne/NeuralHermes-2.5-Mistral-7B. This merge was performed using the slerp (spherical linear interpolation) method, which combines the strengths of its constituent models to enhance overall performance.

Key Capabilities

  • General Language Understanding: The model exhibits robust capabilities across various NLP tasks, as indicated by its performance on the Open LLM Leaderboard.
  • Reasoning: Achieves a score of 67.75 on the AI2 Reasoning Challenge (25-Shot) and 69.75 on GSM8k (5-shot), suggesting proficiency in logical and mathematical reasoning.
  • Context Handling: Supports a context length of 4096 tokens, allowing it to process and generate longer sequences of text.
  • Instruction Following: Inherits instruction-following capabilities from its base models, making it suitable for chat and prompt-based interactions.

Performance Highlights

On the Hugging Face Open LLM Leaderboard, NeuralPipe-7B-slerp achieved an average score of 71.17. Specific benchmark results include:

  • HellaSwag (10-Shot): 86.15
  • MMLU (5-Shot): 63.94
  • TruthfulQA (0-shot): 59.80
  • Winogrande (5-shot): 79.64

Good For

  • General-purpose text generation: Capable of producing coherent and contextually relevant text for various applications.
  • Reasoning tasks: Its scores on reasoning benchmarks suggest suitability for tasks requiring logical inference.
  • Instruction-tuned applications: Can be effectively used in conversational agents or systems requiring adherence to specific prompts.
  • Developers seeking a balanced 7B model: Offers a strong blend of performance and efficiency for a model of its size.

Finetunes
5 models