Overview

pandafish-7b is a 7 billion parameter instruction-tuned model created by ichigoberry. It is built using a "Model Stock" merge method, combining several Mistral-based models: mistralai/Mistral-7B-v0.1, mistralai/Mistral-7B-Instruct-v0.2, CultriX/NeuralTrix-bf16, and OpenPipe/mistral-ft-optimized-1227. This merging strategy aims to synthesize the strengths of its constituent models into a single, capable instruction-following LLM.

Key Capabilities & Performance

The model demonstrates competitive performance across various evaluation benchmarks. While its overall average score is slightly below Mistral-7B-Instruct-v0.2, pandafish-7b shows notable strengths in specific areas:

• AGIEval: Achieves a score of 40, outperforming Mistral-7B-Instruct-v0.2 (38.5).
• GPT4All: Scores 74.23, also surpassing Mistral-7B-Instruct-v0.2 (71.64).

These results suggest that pandafish-7b may excel in tasks related to general knowledge and reasoning, as indicated by its higher scores on AGIEval and GPT4All. The model operates with a context length of 8192 tokens.

Usage

Developers can integrate pandafish-7b using the Hugging Face transformers library, with support for bfloat16 precision and device_map="auto" for efficient deployment.