chargoddard/llama-2-34b-uncode
chargoddard/llama-2-34b-uncode is a 34 billion parameter experimental language model based on the Llama 2 architecture, featuring a 32768 token context length. This model is an early-stage experiment, as indicated by its "very wip experiment" status. Its current performance on the Open LLM Leaderboard suggests it is still under development, with an average score of 36.2 across various benchmarks.
Loading preview...
Overview
chargoddard/llama-2-34b-uncode is a 34 billion parameter experimental language model built upon the Llama 2 architecture, designed with a substantial 32768 token context window. This model is explicitly noted as a "very wip experiment," indicating it is in an early and active development phase.
Current Performance
Evaluations on the Open LLM Leaderboard provide insight into its current capabilities, though these scores reflect its work-in-progress status. Key benchmark results include:
- Avg.: 36.2
- ARC (25-shot): 39.51
- HellaSwag (10-shot): 33.9
- MMLU (5-shot): 38.49
- TruthfulQA (0-shot): 40.94
- Winogrande (5-shot): 74.35
- GSM8K (5-shot): 20.77
- DROP (3-shot): 5.43
Use Cases
Given its experimental nature and current benchmark scores, this model is primarily suited for:
- Research and Development: Exploring the behavior and potential of Llama 2-based models in an experimental setting.
- Early-stage Prototyping: For developers interested in contributing to or testing models still under active development.
It is important to note that its performance metrics suggest it is not yet optimized for production environments or tasks requiring high accuracy across general language understanding, reasoning, or mathematical problem-solving.