startlux-models/StartLux-Decision-v1-9B
StartLux-Decision-v1-9B by StartLux Labs is a 9 billion parameter multimodal decision model designed to answer typed questions about a given state, providing probabilities for each option. It supports text, JSON, and image inputs up to a 262,144 token context length. Optimized for fast inference, it excels at tasks requiring probabilistic decision-making, such as classifying customer support tickets, assessing severity, or evaluating image-based evidence, with a latency of 35.7 ms for three questions on an H200 GPU.
Loading preview...
StartLux-Decision-v1-9B Overview
StartLux-Decision-v1-9B is a 9 billion parameter multimodal model developed by StartLux Labs, specifically engineered for probabilistic decision-making. It processes typed questions about a given state, which can include text, JSON, or images, and returns a probability for each possible option. The model supports an extensive context length of 262,144 tokens (256K), allowing for comprehensive input analysis.
Key Capabilities
- Probabilistic Decision Output: Provides a probability score for each option in response to a question, enabling nuanced decision support.
- Multimodal Input: Accepts diverse input types including text, JSON, and images, with a dedicated vision tower for image processing.
- Extended Context Window: Handles long inputs up to 262,144 tokens, processing large states efficiently.
- High-Speed Inference: Achieves low latency, with 35.7 ms for three questions and 17.6 ms for a single yes/no question on an H200 GPU, thanks to optimized inference packages and CUDA graph utilization.
- TypeSafe API Integration: Uses the
/v1/systemoneformat, ensuring compatibility with existing Jev clients.
Good for
- Automated Triage and Classification: Ideal for categorizing customer support tickets, assigning severity, or routing tasks based on textual or multimodal evidence.
- Probabilistic Assessment: Use cases requiring a confidence score or probability distribution over choices, such as risk assessment or sentiment analysis with graded options.
- Multimodal Data Analysis: Applications that need to integrate information from both text and images to make informed decisions, like evaluating product damage from photos and descriptions.
- High-Throughput Decision Systems: Environments demanding rapid, high-volume decision processing due to its optimized inference speed and batching capabilities.