MOOSE-Star-IR-R1D-7B Overview
MOOSE-Star-IR-R1D-7B (MS-IR-7B) is a 7 billion parameter language model developed by ZonglinY, fine-tuned from the DeepSeek-R1-Distill-Qwen-7B base model. Its primary purpose is to facilitate scientific discovery by identifying cross-paper inspirations, a core component of the MOOSE-Star framework for scientific hypothesis generation.
Key Capabilities
- Specialized Inspiration Retrieval: The model is specifically trained to select the single correct cross-paper inspiration from a set of 15 candidates (one ground truth and 14 hard negatives) based on a given research question and background survey.
- Chain-of-Thought Reasoning: It generates a detailed reasoning process before outputting the selected inspiration ID and its justification.
- Hierarchical Search Integration: Designed to operate within a hierarchical search pipeline, contributing to an O(log N) complexity for scientific discovery tasks.
- Performance: Achieves an accuracy of 54.37% on its specialized inspiration retrieval task, a substantial improvement over the base model's 28.42% and random selection's 6.70%.
Training Details
MS-IR-7B was fine-tuned using full-parameter SFT (ZeRO-3) on the TOMATO-Star-SFT-Data-R1D-32B IR split, comprising over 150,000 training examples. It utilized DeepSeek-R1-Distill-Qwen-32B as a teacher model and was trained for one epoch with a cutoff length of 16384 tokens.
Good For
- Researchers and developers working on automated scientific discovery and hypothesis generation systems.
- Applications requiring intelligent selection of relevant scientific literature for inspiration.
- Integration into complex AI frameworks for scientific research, particularly those leveraging the MOOSE-Star methodology.