MaziyarPanahi/SciPhi-Self-RAG-Mistral-7B-32k-Mistral-7B-Instruct-v0.2-slerp

TEXT GENERATIONConcurrency Cost:1Model Size:7BQuant:FP8Ctx Length:4kPublished:Jan 11, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Cold

MaziyarPanahi/SciPhi-Self-RAG-Mistral-7B-32k-Mistral-7B-Instruct-v0.2-slerp is a 7 billion parameter language model created by MaziyarPanahi, resulting from a slerp merge of Mistral-7B-Instruct-v0.2 and SciPhi-Self-RAG-Mistral-7B-32k. This model integrates the instruction-following capabilities of Mistral-7B-Instruct-v0.2 with the Retrieval-Augmented Generation (RAG) optimizations from SciPhi-Self-RAG-Mistral-7B-32k, making it suitable for tasks requiring both general instruction adherence and enhanced factual grounding. It leverages a 4096 token context length from its base Mistral architecture.

Loading preview...

Overview

This model, named SciPhi-Self-RAG-Mistral-7B-32k-Mistral-7B-Instruct-v0.2-slerp, is a 7 billion parameter language model developed by MaziyarPanahi. It is a product of a 'slerp' (spherical linear interpolation) merge between two distinct base models:

  • mistralai/Mistral-7B-Instruct-v0.2: Known for its strong instruction-following capabilities.
  • SciPhi/SciPhi-Self-RAG-Mistral-7B-32k: Optimized for Self-Retrieval-Augmented Generation (Self-RAG), suggesting enhanced factual accuracy and reduced hallucinations through retrieval mechanisms.

This merging approach aims to combine the strengths of both parent models, integrating robust instruction adherence with advanced RAG capabilities within a 7B parameter footprint.

Key Capabilities

  • Instruction Following: Inherits strong instruction-following abilities from Mistral-7B-Instruct-v0.2.
  • Retrieval-Augmented Generation (RAG): Benefits from the RAG optimizations of SciPhi-Self-RAG-Mistral-7B-32k, which typically improves factual consistency and reduces fabrication by integrating external knowledge.
  • Merged Architecture: Utilizes a slerp merge method, combining layers from both source models to create a hybrid model.

Good For

  • Applications requiring a balance of general instruction-following and factually grounded responses.
  • Scenarios where leveraging external knowledge for more accurate outputs is critical.
  • Developers looking for a 7B model with integrated RAG capabilities without needing to implement RAG externally.

Finetunes
2 models