SciPhi/SciPhi-Mistral-7B-32k
SciPhi/SciPhi-Mistral-7B-32k is a 7 billion parameter Large Language Model (LLM) fine-tuned from Mistral-7B-v0.1 by SciPhi. This model underwent fine-tuning over four epochs using over 1 billion tokens, including instruction tuning data and synthetic textbooks. Its primary objective is to enhance scientific reasoning and educational abilities, making it suitable for tasks requiring advanced academic understanding.
Loading preview...
SciPhi-Mistral-7B-32k: Enhanced Scientific Reasoning
SciPhi-Mistral-7B-32k is a 7 billion parameter language model developed by SciPhi, fine-tuned from the Mistral-7B-v0.1 base model. The fine-tuning process involved over 1 billion tokens across four epochs, incorporating both standard instruction tuning data and synthetic textbooks.
Key Capabilities & Features
- Scientific Reasoning: Specifically optimized to improve performance in scientific and educational contexts.
- Instruction Following: Designed to follow Alpaca prompting guidelines for optimal results.
- Base Architecture: Leverages the robust Mistral-7B-v0.1 architecture, featuring Grouped-Query Attention, Sliding-Window Attention, and a Byte-fallback BPE tokenizer.
Good For
- Educational Applications: Ideal for tasks requiring understanding and generation of academic content.
- Scientific Research Support: Can assist in processing and generating scientific text.
- Instruction-tuned tasks: Benefits from Alpaca-style prompting for various instruction-following scenarios.
This model aims to provide enhanced capabilities for users working with scientific and educational data, building upon the strong foundation of Mistral-7B-v0.1.