SciPhi-Mistral-7B-32k: Enhanced Scientific Reasoning

SciPhi-Mistral-7B-32k is a 7 billion parameter language model developed by SciPhi, fine-tuned from the Mistral-7B-v0.1 base model. The fine-tuning process involved over 1 billion tokens across four epochs, incorporating both standard instruction tuning data and synthetic textbooks.

Key Capabilities & Features

• Scientific Reasoning: Specifically optimized to improve performance in scientific and educational contexts.
• Instruction Following: Designed to follow Alpaca prompting guidelines for optimal results.
• Base Architecture: Leverages the robust Mistral-7B-v0.1 architecture, featuring Grouped-Query Attention, Sliding-Window Attention, and a Byte-fallback BPE tokenizer.

Good For

• Educational Applications: Ideal for tasks requiring understanding and generation of academic content.
• Scientific Research Support: Can assist in processing and generating scientific text.
• Instruction-tuned tasks: Benefits from Alpaca-style prompting for various instruction-following scenarios.

This model aims to provide enhanced capabilities for users working with scientific and educational data, building upon the strong foundation of Mistral-7B-v0.1.