Enoch/llama-7b-hf

What is Enoch/llama-7b-hf?

Enoch/llama-7b-hf is a 7 billion parameter auto-regressive language model, part of the LLaMA family developed by Meta AI's FAIR team. This specific version is a conversion of the original LLaMA-7B model to be compatible with the HuggingFace Transformers library. Trained between December 2022 and February 2023, LLaMA is a foundational model based on the transformer architecture.

Key Capabilities & Characteristics

• Research Focus: Primarily intended for research into large language models, including understanding their capabilities, limitations, and developing improvements.
• Base Model: LLaMA is a base model, meaning it is not fine-tuned with human feedback and may generate toxic, offensive, or incorrect content.
• Multilingual Data: While predominantly English, the training data included 20 languages, though performance is expected to be better for English.
• Evaluation: Evaluated on benchmarks such as MMLU, BIG-bench hard, and various common sense reasoning tasks, showing competitive performance for its size.
• Bias Awareness: The model's training data, sourced from the Web, reflects inherent biases, which have been evaluated using RAI datasets for gender, religion, race, and other categories.

Intended Use Cases

• Exploring Applications: Suitable for research into potential applications like question answering, natural language understanding, and reading comprehension.
• Model Analysis: Ideal for researchers studying the capabilities and limitations of current language models, and for developing techniques to mitigate issues like bias and hallucination.
• Academic Research: Designed for researchers in natural language processing, machine learning, and artificial intelligence.