Hugging Face Overview
Model Overview
ReWiz-Llama-3.2-3B is a 3.2 billion parameter language model developed by Rasmus Rasmussen. This model is built upon the Llama architecture and is designed for a wide range of natural language processing tasks. It supports a substantial context length of 32,768 tokens, allowing it to process and generate longer sequences of text.
Key Capabilities
- General Language Understanding: Capable of processing and interpreting various forms of text.
- Text Generation: Can generate coherent and contextually relevant text.
- Extended Context: Benefits from a 32,768 token context window, useful for tasks requiring extensive memory or long-form content.
Performance Benchmarks
Evaluated on the Open LLM Leaderboard, ReWiz-Llama-3.2-3B shows an average score of 17.98. Specific benchmark results include:
- IFEval (0-Shot): 46.49
- BBH (3-Shot): 19.29
- MATH Lvl 5 (4-Shot): 9.74
- MMLU-PRO (5-shot): 20.97
These scores indicate its performance across instruction following, multi-task language understanding, and mathematical reasoning.
Good For
- Applications requiring a compact yet capable language model.
- Tasks benefiting from a large context window.
- General-purpose text generation and understanding where a 3.2B parameter model is suitable.