NousResearch/Nous-Hermes-Llama2-70b
Nous-Hermes-Llama2-70b is a 69 billion parameter language model developed by Nous Research, fine-tuned on over 300,000 instructions using a 4096 sequence length. This model is notable for its long responses, reduced hallucination rate, and absence of OpenAI censorship mechanisms in its synthetic training data. It was trained primarily on high-quality synthetic GPT-4 outputs from diverse sources, making it suitable for complex instruction following and general language tasks. The model maintains consistency with the original Hermes on Llama-1 while offering enhanced capabilities.
Loading preview...
Nous-Hermes-Llama2-70b Overview
Nous-Hermes-Llama2-70b is a 69 billion parameter instruction-tuned language model developed by Nous Research, with key contributions from Teknium and Emozilla. It was fine-tuned on over 300,000 instructions, utilizing a 4096 token sequence length during training on an 8x H100 80GB machine.
Key Capabilities & Characteristics
- Instruction Following: Fine-tuned on a vast dataset of synthetic GPT-4 outputs, including GPTeacher, Wizard LM, Nous Research Instruct Dataset, GPT4-LLM, Unnatural Instructions, Airoboros, Camel-AI, and CodeAlpaca.
- Reduced Hallucination: Engineered to exhibit a lower rate of hallucination compared to other models.
- Longer Responses: Designed to produce more extensive and detailed outputs.
- Censorship-Free: The synthetic training data used does not incorporate OpenAI's censorship mechanisms.
- Performance: Achieves competitive scores on benchmarks such as the GPT4All Suite (e.g., 0.5734 acc on arc_challenge, 0.8422 acc on boolq) and various BigBench tasks.
Good For
- Applications requiring detailed and lengthy text generation.
- Tasks where adherence to complex instructions is critical.
- Use cases where a model free from specific censorship biases is preferred.
- General language tasks, including creative text generation and understanding nuanced prompts.