SciPhi/SciPhi-Self-RAG-Mistral-7B-32k
SciPhi-Self-RAG-Mistral-7B-32k is a 7 billion parameter Large Language Model developed by SciPhi, fine-tuned from Mistral-7B-v0.1. This model specializes in Retrieval Augmented Generation (RAG) tasks, having undergone further fine-tuning on the self-rag dataset and other RAG-related instruct datasets. It leverages a Transformer architecture with Grouped-Query Attention and Sliding-Window Attention, making it suitable for applications requiring robust information retrieval and generation.
Loading preview...
SciPhi-Self-RAG-Mistral-7B-32k Overview
SciPhi-Self-RAG-Mistral-7B-32k is a 7 billion parameter language model developed by SciPhi, building upon the Mistral-7B-v0.1 base model. Its primary distinction lies in its specialized fine-tuning for Retrieval Augmented Generation (RAG). The model was initially fine-tuned as SciPhi-Mistral-7B-32k and subsequently received further training on the self-rag dataset, incorporating other RAG-related instruct datasets to maintain its conversational tone.
Key Capabilities
- Specialized RAG Performance: Optimized for tasks requiring information retrieval and generation, leveraging the self-rag dataset for enhanced performance.
- Mistral-7B Architecture: Inherits the efficient Transformer architecture of Mistral-7B-v0.1, including Grouped-Query Attention and Sliding-Window Attention.
- Benchmark Performance: The model benchmarks well, indicating strong foundational capabilities, though it is noted that further tuning could enhance its conversational abilities.
Good For
- Applications requiring robust Retrieval Augmented Generation where external knowledge retrieval is crucial.
- Developers looking for a Mistral-based model with specific RAG optimizations.
- Use cases that can benefit from its specialized training on the self-rag dataset.
SciPhi-AI offers a free hosted API where this model is currently available, with more details in their documentation.