hankleetw/llama_3.2_3b_instruct
The hankleetw/llama_3.2_3b_instruct model, developed by Meta, is a 3.21 billion parameter instruction-tuned multilingual large language model from the Llama 3.2 family. Optimized for multilingual dialogue use cases, it excels in agentic retrieval and summarization tasks. This model supports a 32768 token context length and is specifically designed for commercial and research applications, outperforming many open-source and closed chat models on common industry benchmarks.
Loading preview...
Model Overview
hankleetw/llama_3.2_3b_instruct is a 3.21 billion parameter instruction-tuned model from Meta's Llama 3.2 family. This multilingual large language model (LLM) is built on an optimized transformer architecture and has been fine-tuned using supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety. It was trained on a new mix of publicly available online data, incorporating logits from larger Llama 3.1 models, and has a knowledge cutoff of December 2023.
Key Capabilities
- Multilingual Dialogue: Optimized for use cases requiring interaction in multiple languages, with official support for English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
- Agentic Applications: Designed for agentic retrieval and summarization tasks, making it suitable for intelligent assistants and information processing.
- Quantization Support: Includes quantized versions (SpinQuant and QLoRA) that significantly improve inference speed (2.4x-2.6x faster decoding) and reduce model size and memory footprint, ideal for constrained environments like mobile devices.
- Long Context Handling: Supports a substantial context length of 32768 tokens, enabling processing of longer inputs for tasks like long-form QA and summarization.
Good For
- Commercial and Research Use: Intended for a broad range of applications in both commercial products and academic research.
- Chatbots and Virtual Assistants: Excels in assistant-like chat applications due to its instruction-tuned nature and multilingual capabilities.
- Knowledge Retrieval and Summarization: Highly effective for tasks involving extracting and summarizing information from large texts.
- On-Device AI: Quantized versions are particularly well-suited for deployment in environments with limited compute resources, such as mobile AI-powered writing assistants.