Overview
This model, hamxea/Llama-2-7b-chat-hf-activity-fine-tuned-v3, is a 7 billion parameter auto-regressive language model built on the Transformer architecture, originally developed by the FAIR team of Meta AI. It is a Hugging Face conversion of the foundational Llama-7B model, specifically adapted for use with the transformers library (version >= 4.28.0). A key improvement in this version is the reduction of model checkpoints to 2 shards, significantly accelerating loading times compared to previous multi-shard versions.
Key Capabilities & Characteristics
- Architecture: Transformer-based, auto-regressive language model.
- Parameter Count: 7 billion parameters.
- Hugging Face Compatibility: Fully compatible with Hugging Face Transformers, including
LlamaForCausalLM and LlamaTokenizer. - Optimized Loading: Features a reduced number of shards (2 instead of 33) for faster disk loading.
- Multilingual Data: Trained on a dataset including 20 languages, though primarily English-centric, suggesting better performance for English tasks.
Intended Use Cases
This model is primarily intended for research purposes in large language models. Specific research areas include:
- Exploring applications like question answering, natural language understanding, and reading comprehension.
- Understanding the capabilities and limitations of current language models.
- Developing techniques to improve model performance and mitigate issues like biases, toxicity, and hallucinations.
Limitations
As a base, foundational model, Llama-2-7b-chat-hf-activity-fine-tuned-v3 has not been trained with human feedback. Consequently, it may generate toxic or offensive content, incorrect information, or unhelpful answers. It is not suitable for direct deployment in downstream applications without further risk evaluation and mitigation.