jhu-clsp/rank1-7b

Overview

jhu-clsp/rank1-7b is a 7.6 billion parameter reasoning reranker model, developed by JHU CLSP, that enhances information retrieval by employing a novel "test-time compute" approach. Based on the Qwen2.5-7B architecture, this model generates explicit reasoning chains within a <think>...</think> section before determining if a document is relevant to a query. This method allows the model to process complex relevance decisions through logical steps, leading to improved performance across various retrieval tasks.

Key Capabilities

• Reasoning Reranking: Generates internal reasoning chains to justify relevance judgments.
• Binary Relevance Judgment: Outputs a true or false relevance decision for query-document pairs.
• Confidence Scoring: Provides a confidence score based on the logits of the true/false tokens.
• Qwen2.5 Base: Built on the robust Qwen2.5-7B model, offering strong foundational language understanding.
• MTEB Compatibility: Integrates with the MTEB benchmarking framework for evaluation.

Good For

• Information Retrieval Systems: Ideal for improving the accuracy and explainability of document reranking.
• Complex Query Understanding: Excels in scenarios where nuanced relevance decisions are required.
• Research in Reasoning Models: Provides a practical example of test-time compute for enhanced decision-making.

For detailed benchmark results and usage examples, refer to the official paper and the GitHub repository.