PuddleJumper-13b Overview

PuddleJumper-13b is a 13 billion parameter language model developed by totally-not-an-llm. It is a merge of the EverythingLM-V2-13b QLoRa and OpenOrca-Platypus2-13B models, combining their strengths for enhanced performance. The model supports a context length of 4096 tokens and is primarily designed for instruction-following tasks.

Key Capabilities & Performance

This model demonstrates solid performance across a range of benchmarks on the Open LLM Leaderboard. Notable scores include:

• Avg. Score: 50.23
• ARC (25-shot): 58.7
• HellaSwag (10-shot): 81.18
• MMLU (5-shot): 58.25
• TruthfulQA (0-shot): 56.44
• Winogrande (5-shot): 72.77

While its GSM8K (5-shot) score is 3.34 and DROP (3-shot) is 20.93, its overall average indicates a capable general-purpose model. Detailed evaluation results are available on the Open LLM Leaderboard.

Prompt Format

PuddleJumper-13b is primarily intended for use with the Vicuna-short prompt format, which includes a system prompt:

You are a helpful AI assistant.

USER: <prompt>
ASSISTANT:

It also supports other system prompts and is compatible with OpenOrca and Platypus prompt formats, offering flexibility for integration into various applications.