NousResearch/Llama-3.2-1B
NousResearch/Llama-3.2-1B is a 1 billion parameter multilingual large language model developed by Meta, part of the Llama 3.2 collection. This instruction-tuned model, with a 32768-token context length, is optimized for multilingual dialogue use cases such as agentic retrieval and summarization. It utilizes an optimized transformer architecture and is fine-tuned with SFT and RLHF to align with human preferences for helpfulness and safety.
Loading preview...
Model Overview
NousResearch/Llama-3.2-1B is a 1 billion parameter model from Meta's Llama 3.2 family, designed for multilingual text-in/text-out applications. It features an optimized transformer architecture and a substantial 32768-token context length, making it suitable for processing longer inputs.
Key Capabilities
- Multilingual Support: Officially supports English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai, with training on a broader set of languages.
- Instruction-Tuned: Optimized for dialogue, agentic retrieval, and summarization tasks through supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF).
- Performance: Benchmarks show competitive performance in its size class across general, rewriting, summarization, instruction following, math, and reasoning tasks, including multilingual MMLU scores.
- Training: Pretrained on up to 9 trillion tokens of publicly available data, with a knowledge cutoff of December 2023. Incorporates knowledge distillation from larger Llama 3.1 models.
Good For
- Assistant-like Chat Applications: Its instruction-tuned nature makes it ideal for conversational AI.
- Agentic Use Cases: Excels in tasks requiring knowledge retrieval and summarization.
- Constrained Environments: The 1B parameter size is suitable for deployment in environments with limited resources, such as mobile devices, while still requiring system safeguards.