Salesforce/xgen-small-9B-base-r
Salesforce/xgen-small-9B-base-r is a 9 billion parameter pre-trained language model from the xGen-small family, developed by Salesforce. This compact model is designed for enterprise applications, featuring a substantial 128k context length achieved through domain-focused data curation and length-extension techniques. It offers strong performance across general knowledge, reasoning, math, science, and coding tasks, making it suitable for applications requiring long-context understanding at a predictable cost.
Loading preview...
xGen-small 9B Base Model Overview
Salesforce's xGen-small-9B-base-r is a 9 billion parameter pre-trained language model, part of the xGen-small series. This model is engineered for enterprise use, focusing on delivering robust performance with a compact footprint and predictable cost.
Key Capabilities & Features
- Extended Context Length: Achieves an impressive 128k context window, enabling processing of very long documents and complex queries.
- Optimized Training: Benefits from domain-focused data curation, scalable pre-training, and length-extension techniques to enhance long-context performance.
- Balanced Performance: Demonstrates competitive results across a range of benchmarks, including general knowledge (ARC-Challenge: 67.4, Big-Bench Hard: 58.2, HellaSwag: 83.7, MMLU: 71.1), math and science (GPQA: 32.0, GSM8K: 83.2, MATH: 52.5), and coding tasks (HumanEval: 53.9, MBPP: 50.1).
- Research Focus: This specific release is intended for research purposes, supporting academic exploration of compact, enterprise-ready language models.
When to Consider This Model
- Long-Context Applications: Ideal for use cases requiring the processing and understanding of extensive text, such as document analysis, legal review, or detailed technical documentation.
- Enterprise Environments: Designed with enterprise needs in mind, offering a balance of performance and cost-efficiency for business applications.
- Research & Development: Suitable for researchers exploring compact language models, long-context capabilities, and efficient pre-training methodologies.