jhu-clsp/rank1-llama3-8b

TEXT GENERATIONConcurrency Cost:1Model Size:8BQuant:FP8Ctx Length:8kPublished:Feb 18, 2025License:mitArchitecture:Transformer Open Weights Cold

The jhu-clsp/rank1-llama3-8b is an 8 billion parameter reasoning reranker model, based on Llama 3.1 8B, developed by jhu-clsp. It uniquely employs test-time compute to generate explicit reasoning chains within ... sections before making binary relevance judgments for information retrieval tasks. This approach allows the model to break down complex relevance decisions into logical steps, enhancing performance on nuanced topics. Its primary use case is improving the accuracy of information retrieval by providing a more robust and explainable reranking mechanism.

Loading preview...

rank1-llama3-8b: Reasoning Reranker for Information Retrieval

rank1-llama3-8b is an 8 billion parameter model, built upon the Llama 3.1 8B base, designed to enhance information retrieval through a novel reasoning reranking approach. Developed by jhu-clsp, this model distinguishes itself by generating explicit reasoning chains during inference before determining document relevance.

Key Capabilities

  • Explicit Reasoning: Generates a detailed thought process within <think>...</think> tags for each query-document pair, leading to more transparent and robust relevance judgments.
  • Binary Relevance Judgment: Outputs a clear true or false for relevance, accompanied by a confidence score derived from token logits.
  • Improved Performance: Leverages test-time compute to break down complex relevance decisions, demonstrating strong performance on retrieval benchmarks, especially for nuanced topics.
  • Llama 3.1 Base: Benefits from the strong foundational capabilities of the Llama 3.1 8B architecture.

Good for

  • Information Retrieval Reranking: Ideal for tasks requiring precise and explainable document reranking.
  • Complex Query Understanding: Excels in scenarios where relevance depends on intricate logical steps rather than simple keyword matching.
  • Research in Explainable AI: Provides a framework for studying and developing models that articulate their reasoning process in retrieval contexts.
  • MTEB Integration: Compatible with the MTEB benchmarking framework for evaluating retrieval performance.

Finetunes
2 models