AdarshSingh7647/Eklav-7B-Reranker-CotGen
AdarshSingh7647/Eklav-7B-Reranker-CotGen is a 7.6 billion parameter language model based on Qwen/Qwen2.5-7B, fine-tuned for passage reranking tasks using a standard full trace CoT SFT (CotGen) method. This model is designed as a pointwise reranker, similar to Rank1, generating a reasoning trace before outputting a true/false relevance score from logits. It achieves an average nDCG@10 of 26.1 on the BRIGHT dataset, making it suitable for information retrieval and relevance scoring applications.
Loading preview...
Eklav-7B-Reranker-CotGen: A CoT-based Reranker
This model, developed by AdarshSingh7647, is a 7.6 billion parameter language model built upon the Qwen/Qwen2.5-7B base. It is specifically trained for passage reranking tasks using a standard full trace Chain-of-Thought (CoT) Supervised Fine-Tuning (SFT) method, referred to as CotGen. This particular checkpoint serves as a baseline for evaluating improvements in the Eklav training approach, which focuses on teaching a model to continue partial reasoning traces.
Key Capabilities and Features
- Passage Reranking: Optimized for determining the relevance of a passage to a given query.
- Pointwise Reranker: Operates by generating a reasoning trace and then outputting a 'true' or 'false' relevance decision based on final token logits, rather than parsing generated text.
- CoT-based Reasoning: Utilizes Chain-of-Thought for improved reasoning during the reranking process.
- Performance: Achieves an average nDCG@10 of 26.1 on the BRIGHT dataset, demonstrating its effectiveness in ranking tasks.
- Robust Inference: Recommended for use with vLLM and stop strings (e.g.,
</think> true,</think> false) to handle potentially long reasoning traces and prevent degeneration, as opposed to fixed token budgets.
Use Cases
This model is well-suited for applications requiring precise document or passage relevance scoring, such as:
- Information Retrieval Systems: Enhancing search result quality by re-ranking retrieved documents.
- Question Answering Systems: Filtering and ranking candidate passages for answer extraction.
- Content Recommendation: Identifying and prioritizing relevant content based on user queries.