gyanbhambhani/oss-jev-qwen05-decision-10k
The gyanbhambhani/oss-jev-qwen05-decision-10k model is a 0.5 billion parameter Qwen2.5-0.5B-Instruct model fine-tuned by Gyan Bhambhani. It is specifically designed to mimic TypeSafe Jev structured judgments (Choice, Score, Noul) by predicting a single constrained next-token. This model excels at Choice tasks, achieving 82.1% accuracy, and is primarily intended for research and local prototyping of schema-locked decisions without relying on proprietary APIs.
Loading preview...
Overview
The gyanbhambhani/oss-jev-qwen05-decision-10k is a 0.5 billion parameter model, fine-tuned from Qwen2.5-0.5B-Instruct by Gyan Bhambhani. Its core purpose is to replicate TypeSafe Jev's structured judgments (Choice, Score, Noul) through a unique method of predicting a single constrained next-token. This model is part of a size sweep, with this specific checkpoint trained on approximately 10,000 distillation rows.
Key Capabilities
- Structured Decision Mimicry: Designed to output structured judgments (Choice, Score, Noul) by predicting a single token from a predefined set of answer codes.
- High Choice Accuracy: Achieves 82.1% accuracy on Choice tasks, demonstrating strong performance in selecting from discrete options.
- Score Correlation: Shows a strong Pearson correlation of 0.804 for Score level predictions.
- Research & Prototyping: Ideal for local development and research into schema-locked decision-making, offering an alternative to proprietary API calls.
Good For
- Research and Local Prototyping: Developers can use this model to experiment with schema-locked decisions without needing to access a proprietary API.
- Code-Constrained Inference: Suitable for serving applications where the softmax operation is restricted to a specific set of answer codes (e.g., 'A'...'Z', '0'...'9', 'Y'|'N').
- Choice-Based Workloads: Particularly effective for tasks requiring a clear choice among options, where it outperforms larger zero-shot models.
Limitations
It's important to note that this model is not intended for production Noul gates or safety-critical yes/no decisions, as its Noul predictions consistently collapse to 'Y'. It also does not claim parity with Jev's calibration or long context capabilities. Users should re-fit calibration on this checkpoint before shipping calibrated Noul.