Model Overview

ReWiz-Llama-3.2-3B is a 3.2 billion parameter language model developed by Rasmus Rasmussen. This model is built upon the Llama architecture and is designed for a wide range of natural language processing tasks. It supports a substantial context length of 32,768 tokens, allowing it to process and generate longer sequences of text.

Key Capabilities

• General Language Understanding: Capable of processing and interpreting various forms of text.
• Text Generation: Can generate coherent and contextually relevant text.
• Extended Context: Benefits from a 32,768 token context window, useful for tasks requiring extensive memory or long-form content.

Performance Benchmarks

Evaluated on the Open LLM Leaderboard, ReWiz-Llama-3.2-3B shows an average score of 17.98. Specific benchmark results include:

• IFEval (0-Shot): 46.49
• BBH (3-Shot): 19.29
• MATH Lvl 5 (4-Shot): 9.74
• MMLU-PRO (5-shot): 20.97

These scores indicate its performance across instruction following, multi-task language understanding, and mathematical reasoning.

Good For

• Applications requiring a compact yet capable language model.
• Tasks benefiting from a large context window.
• General-purpose text generation and understanding where a 3.2B parameter model is suitable.