sheryc/Qwen2.5-14B-Instruct-CARE: Context-Aware Retrieval-Enhanced Reasoning
This model, based on the 14.7 billion parameter Qwen2.5-14B-Instruct, is enhanced with the CARE (Context-Aware Retrieval-Enhanced reasoning) framework. Its core innovation lies in its native retrieval-augmented reasoning capabilities, designed to significantly improve context fidelity and reduce hallucinations by teaching the model to explicitly integrate in-context evidence into its reasoning process.
Key Capabilities
- Native Retrieval-Augmented Reasoning: Dynamically identifies and incorporates relevant evidence from the input context, leading to more grounded responses.
- Improved Context Fidelity: Demonstrates superior adherence to provided context, even when it contradicts the model's parametric knowledge, minimizing factual errors.
- Enhanced Multi-Hop Reasoning: Excels at complex reasoning tasks that require synthesizing information from multiple pieces of evidence.
- Structured Reasoning Output: Generates transparent reasoning chains, explicitly citing evidence using
<think> and <retrieval> tags, improving explainability. - Two-Phase Training: Utilizes a novel Supervised Fine-Tuning (SFT) phase on HotpotQA for evidence integration patterns, followed by a Reinforcement Learning phase with Group Relative Policy Optimization (GRPO) on datasets like DROP and MS MARCO, optimizing for accuracy, format, and retrieval consistency.
Good For
- Applications requiring high factual accuracy: Ideal for scenarios where grounding responses in provided context is critical.
- Complex question answering: Particularly effective for questions demanding multi-step reasoning and evidence synthesis.
- Explainable AI (XAI): The structured output with explicit citations makes it suitable for use cases where understanding the model's reasoning process is important.
- Reducing hallucinations: Designed to mitigate common LLM issues by enforcing context adherence.