nickprock/archai-jev-qwen-1.5b

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Oct 2, 2026Architecture:Transformer Featherless Exclusive Cold

nickprock/archai-jev-qwen-1.5b is a 1.5 billion parameter Qwen2.5-Instruct model fine-tuned for fast, closed-choice decision-making. It is designed as a "System One" model to provide probabilistic outputs for given options based on log-probabilities of option keys. This model excels at routing customer requests and similar classification tasks, demonstrating improved performance on in-distribution and near out-of-distribution tasks compared to its base model, with a context length of 32768 tokens.

Loading preview...

Overview

nickprock/archai-jev-qwen-1.5b is a 1.5 billion parameter Qwen2.5-Instruct model, specifically fine-tuned as a "System One" model for rapid, closed-choice decision-making. It processes a given state and question with arbitrary options (or yes/no questions), returning a probability for each option based on the log-probabilities of their keys. The model's weights include the merged LoRA adapter, making it ready for immediate use.

Key Capabilities

  • Probabilistic Decision-Making: Provides probabilities for each option in a closed-choice question, derived from log-probabilities of option keys.
  • Optimized for Classification: Designed for tasks like routing customer requests, intent classification, and safety checks.
  • Integration with Decision Index: Uses the input format of the transformers engine from the Decision Index kit, allowing for direct evaluation.
  • GGUF Quantization Available: A Q8_0 quantized version (archai-jev-qwen1.5b-q8.gguf) is provided for llama.cpp for efficient local deployment, supporting a context length of 32,768 tokens.

Performance and Limitations

  • Improved In-Distribution Accuracy: Shows a significant gain of +0.148 in macro accuracy on in-distribution tasks compared to the base model.
  • Modest Near OOD Transfer: Achieves a small but positive transfer of +0.017 on near out-of-distribution tasks (same family, unseen task).
  • Limited Far OOD Transfer: Performance on far out-of-distribution tasks (unseen families) is largely flat (-0.002).
  • Challenges with Many Options: Classification performance remains weak for tasks with a large number of options (e.g., 60 or 150 options), with notable calibration issues.
  • Specific Task Degradation: Performance on paws and cola tasks is worse than the base model, with paws falling below its majority baseline.

Finetunes
1 models