Llama 2 13B: Pretrained Language Model by Meta
This model is the 13 billion parameter pretrained variant from Meta's Llama 2 collection, optimized for general natural language generation. It is an auto-regressive language model built on an optimized transformer architecture.
Key Capabilities & Features
- Parameter Scale: 13 billion parameters, offering a balance between performance and computational requirements.
- Context Length: Supports a context window of 4096 tokens.
- Training Data: Pretrained on 2 trillion tokens from a new mix of publicly available online data, with a data cutoff of September 2022.
- Performance: Demonstrates improved performance over its predecessor, Llama 1, across various academic benchmarks including Code, Commonsense Reasoning, World Knowledge, Reading Comprehension, and MMLU.
- License: Governed by a custom commercial license from Meta, requiring acceptance before download.
Intended Use Cases
- Research and Commercial Use: Designed for both research and commercial applications primarily in English.
- Natural Language Generation: Suitable for adapting to a wide array of natural language generation tasks.
Limitations
- English-only Focus: Primarily intended for use in English; performance in other languages is not guaranteed.
- Potential for Objectionable Content: As with all LLMs, it may produce inaccurate, biased, or otherwise objectionable responses, necessitating safety testing for specific applications.