Intel/neural-chat-7b-v3-1

Intel/neural-chat-7b-v3-1 Overview

Intel/neural-chat-7b-v3-1 is a 7 billion parameter large language model developed by Intel, building upon the mistralai/Mistral-7B-v0.1 architecture. It was fine-tuned using the Direct Performance Optimization (DPO) method on the Open-Orca/SlimOrca dataset, with additional alignment from Intel/orca_dpo_pairs. Training was performed on Intel Gaudi 2 processors, highlighting its optimization for Intel hardware.

Key Capabilities & Performance

• Enhanced Performance: This model shows notable improvements over its base model, Mistral-7B-v0.1, and its predecessor, neural-chat-7b-v3, across various benchmarks on the LLM Leaderboard. For instance, it achieves an average score of 59.06 compared to Mistral-7B-v0.1's 50.32.
• Context Length: Supports an 8192-token context length, consistent with the base Mistral-7B-v0.1 model.
• Quantization Support: Demonstrates amenability to quantization (e.g., INT4) for reduced model size and potentially faster inference, as shown by testing with Intel Extension for Transformers and Intel Extension for PyTorch.

Intended Use Cases

• General Language Tasks: Suitable for a broad range of language-related inference tasks.
• Fine-tuning Base: Can serve as a strong base model for further fine-tuning on specific downstream applications.
• Intel Hardware Optimization: Optimized for deployment and inference on Intel Gaudi 2 processors, with support for NVIDIA GPUs as well.

Limitations

Users should be aware that the model can produce factually incorrect, biased, or offensive outputs. Safety testing is recommended before deployment, and it should not be relied upon for factually accurate information without verification.