h2oai/h2o-lightning-4b
H2O-Lightning-4B is a 4.5 billion parameter decision model from H2O.ai, built on Qwen/Qwen3.5-4B, designed to answer typed decision questions about records and associated images. It returns probabilities for choices, yes/no statements, and ordinal scores. This model excels at decision-making tasks, achieving #1 on JevBench's Composite Score, outperforming larger models in calibration, cost, and latency. It supports a 32768-token context length and runs efficiently on vLLM 0.30.0.
Loading preview...
H2O-Lightning-4B: A Specialized Decision Model
H2O-Lightning-4B is a 4.5 billion parameter model developed by H2O.ai, based on Qwen/Qwen3.5-4B. It is specifically engineered to answer structured decision questions about text records and accompanying images, providing probabilistic outputs for multiple-choice, yes/no, and ordinal scoring tasks. This model is optimized for efficiency, delivering one output token per decision, making it highly cost-effective.
Key Capabilities
- Decision-Making Excellence: Achieved the #1 position on JevBench's Composite Score from October 7–20, 2026, surpassing larger 12B, 26B, and 31B models in overall performance, calibration, cost, and speed.
- Probabilistic Outputs: Provides probabilities for each option in choice, yes/no (noul), and score questions, enabling nuanced decision analysis.
- Multimodal Input: Processes both text records (documents, tickets, conversations) and associated images (photos, screenshots, charts).
- Efficiency: Designed for low-latency, high-throughput decision tasks, with each decision requiring only one forward pass and one output token.
- Flexible Deployment: Runs on unmodified vLLM 0.30.0 and is available in quantized formats like GGUF for llama.cpp and MLX for Apple Silicon.
- Two-way Reading: Features a unique 'two-way' attention mechanism (bidirectional on full-attention layers) for improved accuracy compared to traditional one-way (causal) reading.
Good for
- Automated Decision Systems: Ideal for applications requiring precise, probabilistic answers to structured questions, such as claims processing, customer support routing, and policy adherence checks.
- Image-Based Analysis: Suitable for tasks involving visual data interpretation, like analyzing scanned documents, screenshots, or charts to make decisions.
- Cost-Sensitive Deployments: Its efficiency and small footprint make it a strong candidate for scenarios where cost and latency are critical factors.
- Benchmarking and Evaluation: Proven performance on JevBench makes it a reliable choice for tasks requiring high accuracy and calibration in decision-making.