Yuan-Li-FNLP/R3-RAG-CS-Qwen
Yuan-Li-FNLP/R3-RAG-CS-Qwen is a 7.6 billion parameter language model fine-tuned from Qwen2.5-7B-base. This model is specifically optimized for multi-hop question answering tasks, having been trained on the 2wikimultihopqa, hotpotqa, and musique datasets. It demonstrates a focus on complex information retrieval and synthesis, making it suitable for applications requiring advanced RAG capabilities.
Loading preview...
Yuan-Li-FNLP/R3-RAG-CS-Qwen: Multi-Hop QA Optimized Qwen2.5-7B
This model is a fine-tuned version of the Qwen2.5-7B base model, developed by Yuan-Li-FNLP. It has been specifically optimized for complex question answering scenarios, particularly those requiring multi-hop reasoning and information retrieval.
Key Capabilities
- Multi-Hop Question Answering: Fine-tuned on datasets like 2wikimultihopqa, hotpotqa, and musique, indicating a strong focus on answering questions that require synthesizing information from multiple sources or steps.
- Qwen2.5-7B Foundation: Leverages the robust architecture and capabilities of the Qwen2.5-7B base model, providing a solid foundation for language understanding and generation.
- Performance: Achieved a validation loss of 0.1620 during its final evaluation, with training loss progressively decreasing over 3 epochs.
Training Details
The model was trained with a learning rate of 7e-06, a total batch size of 128 (across 8 GPUs), and utilized the AdamW_Torch optimizer with a cosine learning rate scheduler. The training spanned 3 epochs, with evaluation metrics recorded at regular steps.
Good For
- Applications requiring advanced Retrieval-Augmented Generation (RAG), especially for complex, multi-step queries.
- Research and development in multi-hop question answering systems.
- Tasks where synthesizing information from diverse sources is critical.