ibm-ai-platform/Bamba-9B-1.8T

TEXT GENERATIONPricing:Input $0.431 / Cached $0.0216 / Output $1.12Concurrent Unit Cost:1Model Size:9BQuant:FP8Context Size:32kPublished:Dec 2, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

Bamba-9B-1.8T is a 9 billion parameter decoder-only language model developed by IBM, based on the Mamba-2 architecture. It was trained from scratch using a two-stage approach, with this specific version representing an intermediate checkpoint from the first stage of pretraining on 1.8 trillion tokens. Designed for a wide range of text generation tasks, its Mamba-2 architecture offers an alternative to traditional transformer models, potentially providing different performance characteristics for various applications.

Loading preview...

Overview

Bamba-9B-1.8T is a 9 billion parameter decoder-only language model developed by IBM, built upon the Mamba-2 architecture. This model is an intermediate checkpoint from the first stage of pretraining, having been trained on 1.8 trillion tokens. The full Bamba-9B model undergoes a two-stage training process, initially on 2 trillion tokens from the Dolma v1.7 dataset, followed by an additional 200 billion tokens from a curated high-quality data blend.

Key Capabilities

  • Text Generation: Designed to handle a wide range of text generation tasks.
  • Mamba-2 Architecture: Utilizes the Mamba-2 architecture, offering an alternative to transformer-based models.
  • Quantization Support: Can be quantized to FP8 using fms-model-optimizer for more efficient storage and inference, significantly reducing memory footprint (e.g., from 39.12 GB to 10.83 GB).
  • Hugging Face Integration: Fully integrated with Hugging Face transformers for easy inference and fine-tuning.
  • llama.cpp Compatibility: Preliminary support for running on llama.cpp (CPU-only, work-in-progress).

Good for

  • Exploring Mamba Architecture: Ideal for researchers and developers interested in evaluating the performance and characteristics of Mamba-2 based models for various NLP tasks.
  • Resource-Efficient Deployment: The availability of FP8 quantization makes it suitable for deployment scenarios where memory and inference efficiency are critical.
  • Custom Fine-tuning: Provides a solid base for supervised fine-tuning (SFT) on specific downstream tasks using tools like Hugging Face's TRL SFTTrainer.

Finetunes
2 models