NousResearch/Meta-Llama-3-8B-Alternate-Tokenizer

Model Overview

NousResearch/Meta-Llama-3-8B-Alternate-Tokenizer is an 8 billion parameter model from the Meta Llama 3 family, developed by Meta. It employs an optimized transformer architecture and is pretrained on over 15 trillion tokens of publicly available data, with a knowledge cutoff of March 2023. This specific repository provides an alternate version of Meta-Llama-3-8B, configured for compatibility with the Hermes tokenizer.

Key Capabilities

• General Language Generation: Capable of generating text and code.
• Dialogue Optimization: Instruction-tuned variants are optimized for assistant-like chat use cases.
• Performance: Demonstrates strong performance across various benchmarks, including MMLU (66.6 for base, 68.4 for instruction-tuned) and HumanEval (62.2 for instruction-tuned).
• Scalability: Utilizes Grouped-Query Attention (GQA) for improved inference scalability.
• Context Length: Supports a context length of 8k tokens.

Good For

• Commercial and Research Use: Intended for a broad range of applications in English.
• Assistant-like Chat: Instruction-tuned models are well-suited for dialogue systems.
• Natural Language Generation: Pretrained models can be adapted for various text generation tasks.
• Developers seeking Hermes tokenizer compatibility: This model variant is specifically set up for use with the Hermes tokenizer, offering an alternative to the standard Llama 3 tokenizer.