Redmond-Puffin-13B Overview

Redmond-Puffin-13B is a 13 billion parameter language model developed by Nous Research, built upon the Llama 2 architecture. It stands out as one of the first commercially available Llama 2-based fine-tuned models from Nous Research. The model was trained on a meticulously curated dataset of 3,000 high-quality GPT-4 examples, many of which are long-context, multi-turn conversations designed to leverage Llama 2's 4096 token context window. Additional training data includes specialized subsections from CamelAI's Physics, Chemistry, Biology, and Math datasets.

Key Capabilities

• Extended Context Handling: Fine-tuned extensively on multi-turn conversations reaching the full 4096 token limit, enabling robust long-context understanding and generation.
• Information Recall: Capable of recalling information up to 2023, offering more current knowledge compared to models with earlier cutoff dates.
• Extensive Pretraining: Pretrained on 2 trillion tokens of text, double the amount of many other open LLMs, contributing to its broad knowledge base.
• Improved Dataset Quality: Version 1.3 addresses and resolves dataset errors present in previous Puffin models, enhancing overall output quality.

Prompt Format

The model utilizes the Vicuna ShareGPT prompt format:

### human:

### gpt:

Current Limitations

Some token mismatch and formatting issues have been identified, which may affect current output quality. Nous Research plans to address these in future updates.