ibm-granite/granite-3.0-8b-base
Granite-3.0-8B-Base is an 8.1 billion parameter decoder-only transformer model developed by IBM, trained on 12 trillion tokens across two stages. It supports a 4096-token sequence length and is designed for a variety of text-to-text generation tasks such as summarization, classification, extraction, and question-answering. This base model serves as a foundation for creating specialized applications and supports 12 languages including English, German, and Japanese.
Loading preview...
Model Overview
Granite-3.0-8B-Base is an 8.1 billion parameter decoder-only language model developed by the Granite Team at IBM. It is built on a dense transformer architecture incorporating GQA, RoPE, MLP with SwiGLU, RMSNorm, and shared input/output embeddings. The model underwent a two-stage training process, initially on 10 trillion tokens from diverse domains (web, code, academic, books, math), followed by an additional 2 trillion tokens of high-quality, curated data to enhance performance on specific tasks.
Key Capabilities
- Text-to-Text Generation: Proficient in tasks like summarization, text classification, information extraction, and question-answering.
- Multilingual Support: Supports 12 languages including English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese, with potential for fine-tuning in other languages.
- Foundation Model: Designed to serve as a robust baseline for developing specialized models tailored to specific application scenarios.
Intended Use Cases
This model is suitable for a broad range of generative AI tasks. It can be used as a foundational model for various text-to-text applications. Developers can fine-tune Granite-3.0-8B-Base to create specialized models for particular use cases, leveraging its extensive pre-training on diverse data. Users should be aware that this base model has not undergone safety alignment and may produce problematic outputs, requiring careful ethical consideration and responsible deployment.