alpindale/Llama-2-13b-hf is a 13 billion parameter pretrained generative text model from Meta's Llama 2 family, converted for Hugging Face Transformers. This auto-regressive language model uses an optimized transformer architecture and was trained on 2 trillion tokens of publicly available online data with a 4k context length. It is intended for commercial and research use in English for natural language generation tasks.
Loading preview...
Llama 2 13B Pretrained Model
This model, alpindale/Llama-2-13b-hf, is a 13 billion parameter variant from Meta's Llama 2 collection of large language models, specifically the pretrained version adapted for Hugging Face Transformers. The Llama 2 family includes models ranging from 7B to 70B parameters, with both pretrained and fine-tuned (Llama-2-Chat) variations. This particular model is designed for general natural language generation tasks.
Key Characteristics
- Architecture: Auto-regressive language model utilizing an optimized transformer architecture.
- Training Data: Pretrained on 2 trillion tokens from a new mix of publicly available online data, with a data cutoff of September 2022.
- Context Length: Supports a context length of 4096 tokens.
- Performance: Demonstrates improved performance over Llama 1 13B across various academic benchmarks, including Code (24.5 vs 18.9), Commonsense Reasoning (66.9 vs 66.1), and MMLU (54.8 vs 46.9).
Intended Use Cases
- Commercial and Research: Suitable for a wide range of commercial and research applications.
- Natural Language Generation: Can be adapted for various natural language generation tasks in English.
Meta developed Llama 2 with a focus on helpfulness and safety, and the fine-tuned chat models (not this specific pretrained version) have shown performance comparable to some popular closed-source models in human evaluations.