ibm-granite/granite-3.1-1b-a400m-instruct
Granite-3.1-1B-A400M-Instruct is a 1.3 billion parameter instruction-tuned Mixture-of-Experts (MoE) model developed by the IBM Granite Team, fine-tuned from Granite-3.1-1B-A400M-Base. It features a 32768 token context length and is optimized for long-context tasks such as summarization and question-answering, alongside general instruction following. The model supports 12 languages and is designed for building AI assistants in various domains, including business applications.
Loading preview...
Model Overview
Granite-3.1-1B-A400M-Instruct is a 1.3 billion parameter instruction-tuned Mixture-of-Experts (MoE) model from the IBM Granite Team, built upon the Granite-3.1-1B-A400M-Base. It is specifically fine-tuned using a combination of open-source and synthetic datasets, with a focus on solving long-context problems. The model incorporates supervised finetuning, reinforcement learning for alignment, and model merging techniques.
Key Capabilities
- Long-Context Tasks: Excels in long document summarization and question-answering, leveraging its 32768 token context window.
- Multilingual Support: Supports 12 languages including English, German, Spanish, French, Japanese, and Chinese, with potential for finetuning in additional languages.
- General Instruction Following: Designed for a wide range of tasks such as summarization, text classification, extraction, and RAG.
- Code and Function-Calling: Capable of handling code-related tasks and function-calling scenarios.
Model Architecture
This model utilizes a decoder-only dense transformer architecture, featuring Grouped-Query Attention (GQA), Rotary Position Embeddings (RoPE), and a SwiGLU MLP. It is configured as a Mixture-of-Experts model with 32 experts and a TopK of 8, resulting in 400M active parameters. Training involved 10 trillion tokens on IBM's Blue Vela supercomputing cluster.
Intended Use
Granite-3.1-1B-A400M-Instruct is suitable for developing AI assistants across various domains, particularly business applications requiring robust instruction following and long-context processing. Users should perform safety testing and tuning for specific tasks, especially for non-English use cases, as performance may vary.