RedHatAI/Llama-2-7b-dolphin-open_platypus-pruned_50

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:4kPublished:Mar 15, 2024Architecture:Transformer Featherless Exclusive Cold

RedHatAI/Llama-2-7b-dolphin-open_platypus-pruned_50 is a 7 billion parameter Llama 2 model, developed by Neural Magic and Cerebras, that has been 50% pruned and then fine-tuned for instruction-following tasks. This model leverages a blend of Platypus, Open Orca, and Dolphin datasets, offering efficient inference due to its sparsity. It is designed for general instruction-following applications, providing a balance between performance and computational efficiency.

Loading preview...

Overview

This model, developed by Neural Magic and Cerebras, is a 50% sparse variant of the Llama 2 7B foundational model. It was created by taking a pre-sparsified Llama 2 7B model and then fine-tuning it for instruction-following. The fine-tuning process utilized a blended dataset comprising Open Platypus, 10% Open Orca, and 10% Dolphin datasets, trained for 2 epochs.

Key Characteristics

  • Sparsity: The model is 50% sparse, meaning half of its parameters have been removed, leading to potential efficiencies in inference.
  • Instruction-Following: Specifically fine-tuned to excel at instruction-following tasks.
  • Dataset Blend: Trained on a diverse blend of high-quality instruction datasets (Platypus, Open Orca, Dolphin).
  • Efficient Deployment: Designed for accelerated inference when deployed with specialized tools like nm-vllm or deepsparse.

Performance Benchmarks

While offering efficiency, the model's performance on standard benchmarks shows slight variations compared to the original Llama-2-7b-instruct:

  • MMLU (5-shot): 45.10% (vs. 48.60% for Llama-2-7b-instruct)
  • HellaSwag (10-shot): 78.86% (vs. 79.45%)
  • WinoGrande (5-shot): 72.61% (vs. 75.69%)
  • ARC-c (25-shot): 50.77% (vs. 53.92%)
  • TruthfulQA (0-shot): 44.40% (vs. 43.63%)
  • GSM8K (5-shot): 16.38% (vs. 15.92%)

Use Cases

This model is suitable for applications requiring a Llama 2 7B-class model with a focus on instruction-following, where computational efficiency and reduced resource consumption are important. Its sparse nature makes it a candidate for scenarios where faster inference or deployment on more constrained hardware is desired, particularly for tasks aligned with the instruction datasets it was trained on.

Finetunes
2 models