dwidlee/systemone-lite-0.5b
The dwidlee/systemone-lite-0.5b model is a 0.5 billion parameter decision model based on Qwen/Qwen2.5-0.5B-Instruct, developed by dwidlee. It is specifically fine-tuned for System One-compatible decision-making tasks, utilizing a specialized dataset for training. This model excels at structured decision processes, such as routing, classification, and scoring, making it suitable for local experiments and demonstrations in constrained environments. It features a context length of 32768 tokens and is optimized for option-restricted next-token scoring.
Loading preview...
Model Overview
dwidlee/systemone-lite-0.5b is a 0.5 billion parameter decision model, built upon the Qwen/Qwen2.5-0.5B-Instruct base. It is specifically designed to be compatible with the System One framework, focusing on structured decision-making tasks. The model was fine-tuned using the dwidlee/systemone-lite-phase2 dataset over 10,000 steps, employing a stratified training approach.
Key Capabilities
- System One Compatibility: Engineered for integration with System One-compatible decision processes.
- Structured Decision Making: Excels at tasks requiring closed-criteria responses, such as yes/no questions, scoring, and routing based on predefined options.
- Specialized Training: Training included specific optimizations for tasks like chess piece movement, spatial gym puzzles (Connect4, Sokoban), and various classification problems.
- Local Deployment: Intended for local experiments and demonstrations due to its compact size (0.5B parameters).
Performance Highlights
- Achieved 61.6% on the Phase2 held-out test set (n=4700) and 63.9% on the First-800 protocol.
- On the public JevBench (231 tasks), it scored 50.7% accuracy with an ECE of 0.245.
- Typical inference latency is 10–30 ms on consumer GPUs for short payloads.
Limitations
- Scale: At 0.5B parameters, it is primarily for demos and local experiments, not production-grade decision services.
- Calibration: Probability outputs are not reliably calibrated for confidence.
- Not Jev: Differs from the original Jev model in weights, scoring path, and confidence formula.
- Spatial Planning: Performance in complex spatial planning tasks (e.g., Sokoban, Chess) is still developing.
Use Cases
This model is ideal for developers and researchers looking to experiment with System One-compatible decision models locally. It can be used for:
- Rapid prototyping of decision-making agents.
- Exploring structured classification and routing tasks.
- Educational purposes to understand fine-tuning for specific decision frameworks.