What is Enoch/llama-7b-hf?
Enoch/llama-7b-hf is a 7 billion parameter auto-regressive language model, part of the LLaMA family developed by Meta AI's FAIR team. This specific version is a conversion of the original LLaMA-7B model to be compatible with the HuggingFace Transformers library. Trained between December 2022 and February 2023, LLaMA is a foundational model based on the transformer architecture.
Key Capabilities & Characteristics
- Research Focus: Primarily intended for research into large language models, including understanding their capabilities, limitations, and developing improvements.
- Base Model: LLaMA is a base model, meaning it is not fine-tuned with human feedback and may generate toxic, offensive, or incorrect content.
- Multilingual Data: While predominantly English, the training data included 20 languages, though performance is expected to be better for English.
- Evaluation: Evaluated on benchmarks such as MMLU, BIG-bench hard, and various common sense reasoning tasks, showing competitive performance for its size.
- Bias Awareness: The model's training data, sourced from the Web, reflects inherent biases, which have been evaluated using RAI datasets for gender, religion, race, and other categories.
Intended Use Cases
- Exploring Applications: Suitable for research into potential applications like question answering, natural language understanding, and reading comprehension.
- Model Analysis: Ideal for researchers studying the capabilities and limitations of current language models, and for developing techniques to mitigate issues like bias and hallucination.
- Academic Research: Designed for researchers in natural language processing, machine learning, and artificial intelligence.