lemiao/decapoda-research-llama-7b-hf
The lemiao/decapoda-research-llama-7b-hf is a 7 billion parameter LLaMA v1 auto-regressive language model, developed by Meta AI's FAIR team and converted for HuggingFace. Trained on 1 trillion tokens with a 4096-token context length, it is primarily intended for research in large language models, focusing on understanding capabilities, limitations, and mitigating biases. This foundational model excels in common sense reasoning and natural language understanding tasks.
Loading preview...
Model Overview
This is a HuggingFace-converted version of Meta AI's FAIR team's LLaMA (Large Language Model Meta AI) v1, a 7 billion parameter auto-regressive language model based on the transformer architecture. Trained between December 2022 and February 2023, the model was developed primarily for research purposes, including exploring applications like question answering and natural language understanding, and evaluating model biases and limitations.
Key Capabilities
- Foundational Research: Designed for researchers to study large language models, their capabilities, and limitations.
- Common Sense Reasoning: Evaluated on benchmarks such as BoolQ, PIQA, SIQA, HellaSwag, and WinoGrande.
- Natural Language Understanding: Performance measured on tasks like MMLU and reading comprehension.
- Multilingual Data: While predominantly English, the training data included 20 languages, suggesting some multilingual understanding.
Intended Use Cases
- Academic Research: Ideal for researchers in NLP, ML, and AI to explore model behaviors, biases, and develop improvements.
- Bias Evaluation: Useful for evaluating and mitigating biases, risks, and toxic content generation in LLMs.
Limitations
As a base model, LLaMA v1 has not been fine-tuned with human feedback and may generate toxic, offensive, or incorrect information. It is not intended for direct deployment in downstream applications without further risk assessment and mitigation.