Model Overview
pankajmathur/orca_mini_v3_7b is a 7 billion parameter language model built upon the Llama 2 architecture, fine-tuned by Pankaj Mathur. Its training incorporates Orca-style datasets, which are known for emphasizing complex explanation traces, aiming to enhance the model's instruction-following capabilities.
Key Capabilities & Performance
This model is designed to be an effective AI assistant, excelling at following instructions. Evaluation results from the HuggingFace Open LLM Leaderboard indicate its performance across several benchmarks:
- ARC Challenge: 57.17% (acc_norm)
- HellaSwag: 79.66% (acc_norm)
- MMLU: 52.34% (acc_norm)
- TruthfulQA: 50.29% (mc2)
These metrics suggest a balanced capability across common reasoning, common sense, and knowledge-based tasks for its size class.
Use Cases
This model is particularly well-suited for:
- Instruction Following: Generating responses based on explicit user instructions.
- General Chatbot Applications: Serving as an interactive AI assistant.
- Exploration of Open-Source LLMs: Providing a performant 7B parameter option for developers interested in Llama 2 derivatives with enhanced instruction-following.
Limitations
As with many LLMs, orca_mini_v3_7b may occasionally produce inaccurate or misleading information. There is also a possibility of generating inappropriate, biased, or offensive content, necessitating careful usage and cross-verification of outputs.