KnutJaegersberg/MistralInstructLongish Overview

KnutJaegersberg/MistralInstructLongish is a 7 billion parameter language model, fine-tuned for instruction following. The model underwent training for approximately three epochs on a composite dataset comprising various instruction datasets, notably including those with extended instruction lengths. This training approach aims to improve its capacity to understand and execute complex instructions.

Key Capabilities & Training

• Instruction Following: Optimized to respond to instructions, utilizing an Alpaca-style prompt format (### Instruction:, ### Input:, ### Response:).
• Dataset Focus: Trained on a merger of several instruction datasets, with an emphasis on incorporating longer instructions to enhance comprehension of detailed prompts.

Performance Benchmarks

Evaluated on the Open LLM Leaderboard, the model achieved an average score of 48.99. Specific benchmark results include:

• ARC (25-shot): 60.75
• HellaSwag (10-shot): 81.86
• MMLU (5-shot): 60.49
• TruthfulQA (0-shot): 40.55
• Winogrande (5-shot): 76.56
• GSM8K (5-shot): 1.52
• DROP (3-shot): 21.22

Good For

• Applications requiring a 7B model capable of following detailed instructions.
• Tasks where an Alpaca-style prompt format is preferred or can be easily adapted.