startlux-models/StartLux-Decision-v1-2B
StartLux-Decision-v1-2B by startlux-models is a 2.3 billion parameter model designed for structured decision-making tasks, providing probabilistic answers to typed questions (choice, yes/no, rating) about a given state. It supports multimodal inputs including text, JSON, and images, with a large context window of up to 262,144 tokens. Optimized for fast inference, it processes requests efficiently, making it suitable for applications requiring rapid, probabilistic decision outputs.
Loading preview...
StartLux-Decision-v1-2B Overview
StartLux-Decision-v1-2B is a 2.3 billion parameter model developed by startlux-models, specifically engineered for structured decision-making. It processes typed questions (e.g., multiple choice, yes/no, rating scales) about a given state and returns a probability for each possible option. This model is part of a family of StartLux-Decision models, ranging from 0.8B to 35B parameters.
Key Capabilities
- Probabilistic Decision Outputs: Provides a probability distribution for each option in response to a question, enabling nuanced decision support.
- Multimodal Input Support: Accepts diverse input formats including text, JSON, and images, allowing for comprehensive state representation.
- Extended Context Window: Handles long inputs up to 262,144 tokens (256K), processing complex states efficiently.
- Fast Inference: Optimized for low-latency inference, achieving 15.5 ms for a request with three questions and 9.6 ms for a single yes/no question on an H200 GPU, utilizing custom kernels and CUDA graphs.
- TypeSafe API Compatibility: Uses the TypeSafe
/v1/systemoneformat, ensuring seamless integration with existing Jev clients. - Batch Processing: Supports
decide_batchfor efficient processing of multiple requests simultaneously.
Good For
- Applications requiring rapid, probabilistic answers to structured questions.
- Systems that need to analyze multimodal data (text, JSON, images) for decision support.
- Use cases where a large context window is crucial for understanding complex scenarios.
- Integrating with existing TypeSafe API-compatible infrastructures.
- Scenarios demanding high throughput and low latency for decision inference.