Model Overview

This model, developed by Liaoyaqi Wang, Zhengping Jiang, Anqi Liu, and Benjamin Van Durme, is a decoding-based regression model (classification) with 14.8 billion parameters, fine-tuned from Qwen/Qwen2.5-14B-Instruct. Its core innovation lies in providing fine-grained conditional probability estimations, moving beyond traditional classification to offer more detailed likelihood scores.

Key Capabilities

• Fine-grained Probability Estimation: Unlike standard classifiers, this model outputs nuanced probability scores for given conditions, as detailed in the associated paper: Always Tell Me The Odds: Fine-grained Conditional Probability Estimation.
• Decoding-based Regression: It leverages a decoding-based approach to convert model logits into meaningful scores, allowing for a more continuous and granular output.
• Customizable Scoring Function: The model integrates with a LevelToScorePipeline in Hugging Face Transformers, enabling users to define custom functions (e.g., _level_to_score_func) to transform raw logits into desired probability scores or expectations.
• Augmented Tokenizer: It utilizes a SingleLabelRankDict to augment the tokenizer's vocabulary with special tokens (e.g., <|label_level_0|>) that represent different probability levels, facilitating the regression task.

Use Cases

• Conditional Probability Assessment: Ideal for scenarios where understanding the likelihood of an event given a premise is crucial, rather than just a binary or multi-class prediction.
• Nuanced Decision Making: Can be applied in systems requiring more detailed confidence levels or risk assessments based on textual inputs.
• Research in Probabilistic AI: Provides a framework for exploring and implementing fine-grained probability estimation using large language models.