startlux-models/StartLux-Decision-v1-27B
StartLux-Decision-v1-27B by startlux-models is a 27 billion parameter decision-making model designed to answer typed questions about a given state, providing probabilities for each option. It supports diverse input types including text, JSON, and images, with a substantial context length of up to 262,144 tokens. This model excels at tasks requiring probabilistic decision outputs like classification, yes/no questions, and rating scales, and is optimized for fast inference with specialized kernels and batch processing capabilities.
Loading preview...
StartLux-Decision-v1-27B Overview
StartLux-Decision-v1-27B is a 27 billion parameter model developed by startlux-models, specifically engineered for probabilistic decision-making. It processes typed questions about a given state (text, JSON, or images) and returns a probability for each possible option. The model supports an extensive context length of up to 262,144 tokens, allowing for comprehensive analysis of long inputs.
Key Capabilities
- Probabilistic Decision Outputs: Provides a probability for each option in response to questions like 'pick one of several options', 'yes or no', or 'rating on a scale'.
- Multi-modal Input: Accepts text, JSON, and images as input, with a vision tower integrated for image processing.
- Extended Context Window: Handles inputs up to 262,144 tokens, enabling detailed analysis of large documents or complex states.
- Fast Inference: Utilizes specialized kernels (
flash-linear-attention,causal-conv1d) and CUDA graph replays for optimized, low-latency inference, achieving 102.3 ms for three questions and 50.7 ms for a single yes/no question on an H200 GPU. - Batch Processing: Features
decide_batchfor efficient processing of multiple requests simultaneously. - TypeSafe API Compatibility: Uses the
/v1/systemoneformat, ensuring compatibility with clients designed for Jev.
Good For
- Automated Triage and Classification: Assigning tickets to teams, categorizing customer inquiries, or determining urgency with probabilistic confidence.
- Risk Assessment and Scoring: Evaluating severity, impact, or likelihood on a defined scale.
- Image-based Decision Making: Answering questions based on visual evidence, such as identifying damage in a parcel image.
- Applications Requiring Probabilistic Confidence: Use cases where not just a decision, but the confidence level of that decision, is crucial.