Overview
AITeamVN/Vi-Qwen2-3B-RAG is a 3.1 billion parameter large language model, fine-tuned from the Qwen2-Instruct base model, specifically designed for Retrieval Augmented Generation (RAG) tasks in Vietnamese. It aims to significantly improve Vietnamese language processing and RAG performance, supporting a context length of up to 8192 tokens. While specialized for RAG, it can also function as a general-purpose chatbot, handling continuous questions within an input context.
Key Capabilities
- Noise Robustness: Effectively extracts useful information from noisy documents, even with mixed positive and negative contexts.
- Negative Rejection: Accurately declines to answer questions when the necessary knowledge is not present in the retrieved documents.
- Information Integration: Answers complex questions by synthesizing information from multiple documents.
- Context Identification: Determines with approximately 99% accuracy whether a context contains the answer to a given question.
- Multipurpose: Can be used for standard chatbot interactions and continuous questioning within a provided context, beyond its primary RAG focus.
Limitations
Due to its specialized RAG design, the model may have limitations regarding:
- Accuracy on questions related to politics or social issues.
- Potential for exhibiting biases or inappropriate viewpoints.
Benchmarks
The model's RAG performance was evaluated using a manually created Vietnamese dataset, EvalRAGData, with human scoring. Additionally, it was benchmarked on the VMLU leaderboard, achieving an average score of 56.04 across various categories.