tttt111/Ministral-8B-Instruct-test
The Ministral-8B-Instruct-2410 is an 8 billion parameter instruct fine-tuned causal language model developed by Mistral AI. It features a 128k context window with interleaved sliding-window attention and is trained on a large proportion of multilingual and code data. This model significantly outperforms existing models of similar size in knowledge, commonsense, code, math, and multilingual benchmarks, and supports function calling, making it suitable for local intelligence, on-device computing, and edge use cases.
Loading preview...
Ministral-8B-Instruct-2410 Overview
Ministral-8B-Instruct-2410 is an 8 billion parameter instruct fine-tuned model from Mistral AI, designed for local intelligence and on-device applications. It is released under the Mistral Research License, with commercial use requiring direct contact with Mistral AI.
Key Features & Capabilities
- Architecture: Dense Transformer with 8.02 billion parameters, 36 layers, and 128k context window.
- Advanced Attention: Utilizes interleaved sliding-window attention for efficient long-context processing.
- Multilingual & Code Training: Trained on extensive multilingual and code datasets, enhancing its performance in these domains.
- Function Calling: Supports function calling, enabling integration with external tools and APIs.
- Tokenizer: Employs a 131k vocabulary size with the V3-Tekken tokenizer.
Performance Highlights
Ministral-8B-Instruct-2410 demonstrates strong performance across various benchmarks, often outperforming Mistral 7B and Llama 3.1 8B:
- Knowledge & Commonsense: Achieves 65.0 on MMLU, 48.3 on AGIEval, and 75.3 on Winogrande.
- Code & Math: Scores 34.8 on HumanEval pass@1 and 64.5 on GSM8K maj@8 for base models, and 76.8 on HumanEval pass@1 and 54.5 on Math maj@1 for instruct models.
- Multilingual: Shows superior performance in French, German, and Spanish MMLU benchmarks.
- Chat/Arena: Achieves an MTBench score of 8.3 and an Arena Hard score of 70.9, as judged by gpt-4o.
- Function Calling: Records an internal benchmark score of 31.6, indicating robust tool use capabilities.
Usage Recommendations
For optimal performance, Mistral AI recommends using this model with the vLLM library for production inference, or mistral-inference for quick testing and experimentation. Note that vLLM currently caps context at 32k, with full 128k context support under development.