decompute/Nebula-S-v1
decompute/Nebula-S-v1 is a 4-billion parameter language model developed by decompute, featuring a Structured-Vector Multi-Stream (SVMS) architecture. This model integrates a multi-stream reasoning layer on top of a frozen backbone, enhancing its capabilities in complex reasoning tasks. With a 32768-token context length, Nebula-S-v1 demonstrates strong performance on mathematical reasoning and advanced knowledge benchmarks like GSM8K and MMLU-Pro.
Loading preview...
Nebula-S-v1: Reasoning-Enhanced Language Model
Nebula-S-v1 is a 4-billion parameter language model developed by decompute, distinguished by its Structured-Vector Multi-Stream (SVMS) architecture. This innovative design incorporates a multi-stream reasoning layer over a frozen 4B-parameter backbone, with only 400 million parameters being trainable. The SVMS architecture includes structured consistency for cross-stream coherence, a learned router for per-token stream weighting, and delta logits for prediction correction.
Key Capabilities & Performance
Nebula-S-v1 was specifically trained on 200,000 Orca Math Word Problems using an adapter-only method, focusing on enhancing its reasoning abilities. Evaluation results highlight its strong performance, particularly in areas requiring logical deduction and advanced knowledge:
- GSM8K: Achieved 90% accuracy, indicating robust mathematical reasoning.
- GPQA: Scored 70.5%.
- HMMT (November 2025): Reached 67%.
- MMLU-Pro: Demonstrated 79.7% performance.
These scores underscore its effectiveness in complex problem-solving and competitive-level tasks, making it a powerful tool for applications requiring precise and structured reasoning.
Good For
- Mathematical Reasoning: Excels in solving math word problems and similar quantitative tasks.
- Complex Problem Solving: Suitable for applications requiring advanced logical deduction and structured thinking.
- Knowledge-Intensive Tasks: Performs well on benchmarks assessing broad knowledge and understanding.