4bit/Redmond-Puffin-13B
Redmond-Puffin-13B is a 13 billion parameter Llama 2-based language model developed by Nous Research, fine-tuned on 3,000 high-quality GPT-4 examples with a 4096 token context length. It excels in multi-turn conversations and long-context recall, with pretraining on 2 trillion tokens. This model is designed for applications requiring extensive conversational memory and up-to-date information recall.
Loading preview...
Redmond-Puffin-13B Overview
Redmond-Puffin-13B is a 13 billion parameter language model developed by Nous Research, built upon the Llama 2 architecture. It stands out as one of the first commercially available Llama 2-based fine-tuned models from Nous Research. The model was trained on a meticulously curated dataset of 3,000 high-quality GPT-4 examples, many of which are long-context, multi-turn conversations designed to leverage Llama 2's 4096 token context window. Additional training data includes specialized subsections from CamelAI's Physics, Chemistry, Biology, and Math datasets.
Key Capabilities
- Extended Context Handling: Fine-tuned extensively on multi-turn conversations reaching the full 4096 token limit, enabling robust long-context understanding and generation.
- Information Recall: Capable of recalling information up to 2023, offering more current knowledge compared to models with earlier cutoff dates.
- Extensive Pretraining: Pretrained on 2 trillion tokens of text, double the amount of many other open LLMs, contributing to its broad knowledge base.
- Improved Dataset Quality: Version 1.3 addresses and resolves dataset errors present in previous Puffin models, enhancing overall output quality.
Prompt Format
The model utilizes the Vicuna ShareGPT prompt format:
### human:
### gpt:Current Limitations
Some token mismatch and formatting issues have been identified, which may affect current output quality. Nous Research plans to address these in future updates.