FreedomIntelligence/RAG-Instruct-Llama3-3B
FreedomIntelligence/RAG-Instruct-Llama3-3B is a 3.2 billion parameter Llama 3-based causal language model developed by FreedomIntelligence. It is specifically trained on RAG-Instruct data, a diverse dataset generated using five RAG paradigms and instruction simulation. This model significantly enhances Retrieval-Augmented Generation (RAG) capabilities, demonstrating improved performance across various RAG tasks compared to its base Llama 3.2-3B counterpart. It is optimized for scenarios requiring robust information retrieval and synthesis from provided contexts.
Loading preview...
Overview
FreedomIntelligence/RAG-Instruct-Llama3-3B is a 3.2 billion parameter language model built upon the Llama 3.2-3B architecture. Its core innovation lies in its training methodology: it is fine-tuned using the RAG-Instruct dataset, a synthetic instruction dataset designed to enhance Retrieval-Augmented Generation (RAG) abilities in large language models.
Key Capabilities & Training
The RAG-Instruct dataset is generated through a novel method that incorporates:
- Five diverse RAG paradigms: These represent various query-document relationships, aiming to improve the model's generalization across different RAG tasks.
- Instruction simulation: This technique enriches the diversity and quality of instructions by leveraging existing instruction datasets.
This specialized training significantly boosts the model's performance in RAG scenarios. Benchmarks show notable improvements over the base Llama 3.2-3B model across a range of question-answering tasks, including WQA, PQA, TQA, OBQA, Pub, ARC, 2WIKI, HotP, MSQ, CFQA, and PubMed. For instance, it achieves 65.3% accuracy on WQA compared to 58.7% for the base model, and 81.2% EM on OBQA versus 77.0%.
Use Cases
This model is particularly well-suited for applications where robust RAG performance is critical. Developers can leverage it for tasks requiring accurate information extraction, synthesis, and question answering based on provided documents or contexts. Its enhanced RAG capabilities make it a strong candidate for building intelligent search, knowledge retrieval, and conversational AI systems that rely on external data sources.