pankajmathur/model_420
Pankaj Mathur's model_420 is a 69 billion parameter Llama2-based causal language model, fine-tuned on Orca-style datasets. This model is optimized for instruction following and general reasoning tasks, achieving an average score of 58.41 on the Open LLM Leaderboard benchmarks. It is suitable for applications requiring robust conversational AI and complex instruction adherence, with a context length of 32768 tokens.
Loading preview...
Overview
pankajmathur/model_420 is a 69 billion parameter language model built upon the Llama2 architecture, specifically fine-tuned using Orca-style datasets. This training approach aims to enhance the model's ability to follow complex instructions and engage in detailed reasoning, making it a strong candidate for advanced conversational AI applications.
Key Capabilities & Performance
The model demonstrates solid performance across various benchmarks, as evaluated on the HuggingFaceH4 Open LLM Leaderboard. It achieved an average score of 58.41, with notable results including 70.14 on ARC, 87.73 on HellaSwag, and 70.35 on MMLU. These scores indicate its proficiency in common sense reasoning, reading comprehension, and general knowledge tasks. The model supports a substantial context length of 32768 tokens, allowing for processing and generating longer, more coherent texts.
Usage Considerations
This model requires significant computational resources, specifically up to 45GB of GPU VRAM for 4-bit loading. It is compatible with setups like a single high-end GPU (e.g., RTX 6000, L40, A40, A100, H100) or dual consumer-grade GPUs (e.g., RTX 4090, L4, A10, RTX 3090, RTX A5000). Users can easily integrate it with the Oobabooga Web UI or through standard Hugging Face Transformers code for generation tasks. While designed for accuracy, users should be aware of potential limitations regarding occasional inaccuracies or biases, as with any large language model.