Model Overview
macadeliccc/Mistral-7B-v0.2-OpenHermes is a 7 billion parameter language model, fine-tuned by macadeliccc from the alpindale/Mistral-7B-v0.2 base model. It was trained using the teknium/OpenHermes-2.5 dataset, with a focus on efficient training processes facilitated by Unsloth and Huggingface's TRL library, completing in 13 hours on an A100 GPU.
Key Capabilities & Features
- RAG Proficiency: The model is specifically noted for its proficiency in Retrieval Augmented Generation (RAG) use cases, making it suitable for applications where external knowledge retrieval is crucial.
- Efficient Training: Achieved rapid training times, indicating optimized fine-tuning techniques.
- ChatML Prompt Template: Utilizes the ChatML format for conversational interactions, ensuring compatibility with common chat interfaces.
- Quantization Support: Available in various quantized formats including GGUF, AWQ, HQQ-4bit, and ExLlamaV2 for optimized deployment across different hardware.
Performance Benchmarks
Evaluations conducted by Maxime Labonne show the model achieving an average score of 45.26% across several benchmarks:
- AGIEval: 35.57%
- GPT4All: 67.15%
- TruthfulQA: 42.06%
- Bigbench: 36.27%
Recommended Use Cases
- RAG Applications: Ideal for scenarios requiring robust information retrieval and generation, with a recommendation for further RAG-specific fine-tuning.
- Conversational AI: Suitable for building helpful assistant chatbots, leveraging its ChatML compatibility.
- Resource-Constrained Environments: The availability of various quantizations makes it adaptable for deployment on hardware with limited resources.