Overview

jhu-clsp/rank1-32b is a 32.8 billion parameter reasoning reranker model, developed by jhu-clsp, that leverages test-time compute for information retrieval. Based on the Qwen2.5-32B architecture, this model is designed to "think" before making relevance judgments, a novel approach detailed in its associated paper.

Key Capabilities

• Reasoning Reranking: Unlike traditional rerankers, rank1-32b generates explicit reasoning chains within a <think>...</think> section for each query-document pair before outputting a binary relevance judgment (true or false).
• Confidence Scoring: It provides a confidence score for its relevance judgment, derived from the logits of the true/false tokens.
• Improved Relevance Decisions: By breaking down complex relevance decisions into logical steps, the model aims to enhance performance across diverse retrieval tasks.
• Scalable Family: Part of a larger family of models ranging from 0.5B to 32B parameters, including variants based on Mistral and Llama 3.1, and quantized versions for efficiency.

Good For

• Information Retrieval: Specifically designed for reranking documents based on their relevance to a query.
• Complex Query Understanding: Ideal for scenarios where nuanced understanding and logical reasoning are required to determine document relevance.
• Research and Development: Useful for researchers exploring test-time compute and reasoning in retrieval systems, with associated training data and run files available.
• MTEB Benchmarking: Compatible with the MTEB framework for evaluating retrieval performance.