castorini/rank_zephyr_7b_v1_full

RankZephyr 7B V1 Full: Specialized Listwise Reranking Model

RankZephyr 7B V1 Full is a 7 billion parameter language model developed by Castorini, built upon the Zephyr-7B-β architecture. It is specifically fine-tuned for listwise reranking, a task where the model reorders a list of documents based on their relevance to a given query. This model represents the "Full" version, which undergoes an additional fine-tuning stage using RankGPT-4 reorderings of OpenAI's Ada2 orderings for 5,000 queries, building upon the initial fine-tuning of RankZephyr Base on RankGPT-3.5 data.

Key Capabilities

• State-of-the-Art Reranking: Achieves leading open-source performance on datasets such as DL19, DL20, DL21, DL22, TREC-COVID, and TREC-News. For instance, with SPLADE++ ED as the first stage, RankZephyr-7b-v1-full-rho scores 0.7855 on DL19 and 0.8255 on DL20, outperforming RankGPT-4.
• Specialized Agent: Functions as a dedicated listwise reranking agent, taking a query and documents to return a reordered list of document identifiers.
• English-centric: Primarily trained on monolingual English data, with effectiveness on multilingual sets not guaranteed.

Intended Use Cases

• Information Retrieval Systems: Ideal for integrating into search engines or recommendation systems to improve the relevance of retrieved results.
• Research and Development: Useful for researchers exploring advanced reranking techniques and comparing performance against other models.

Limitations

• Inherits potential biases and limitations from its base model, Zephyr-7B-β, which was not aligned to human preferences for safety via RLHF.
• Performance on multilingual datasets is not guaranteed due to its English-centric training.