jhu-clsp/rank1-32b
The jhu-clsp/rank1-32b model is a 32.8 billion parameter reasoning reranker, built upon the Qwen2.5-32B base model, designed for information retrieval tasks. It uniquely employs test-time compute to generate explicit reasoning chains before making relevance judgments for query-document pairs. This approach allows the model to break down complex relevance decisions into logical steps, enhancing performance on tasks requiring nuanced understanding. It is specifically optimized for improving the accuracy of information retrieval by providing confidence scores for relevance.
Loading preview...
Overview
jhu-clsp/rank1-32b is a 32.8 billion parameter reasoning reranker model, developed by jhu-clsp, that leverages test-time compute for information retrieval. Based on the Qwen2.5-32B architecture, this model is designed to "think" before making relevance judgments, a novel approach detailed in its associated paper.
Key Capabilities
- Reasoning Reranking: Unlike traditional rerankers, rank1-32b generates explicit reasoning chains within a
<think>...</think>section for each query-document pair before outputting a binary relevance judgment (trueorfalse). - Confidence Scoring: It provides a confidence score for its relevance judgment, derived from the logits of the true/false tokens.
- Improved Relevance Decisions: By breaking down complex relevance decisions into logical steps, the model aims to enhance performance across diverse retrieval tasks.
- Scalable Family: Part of a larger family of models ranging from 0.5B to 32B parameters, including variants based on Mistral and Llama 3.1, and quantized versions for efficiency.
Good For
- Information Retrieval: Specifically designed for reranking documents based on their relevance to a query.
- Complex Query Understanding: Ideal for scenarios where nuanced understanding and logical reasoning are required to determine document relevance.
- Research and Development: Useful for researchers exploring test-time compute and reasoning in retrieval systems, with associated training data and run files available.
- MTEB Benchmarking: Compatible with the MTEB framework for evaluating retrieval performance.