sascha-frank-ai-research/tsft-rag-qwen2.5-0.5b-instruct
The sascha-frank-ai-research/tsft-rag-qwen2.5-0.5b-instruct is a 0.5 billion parameter Qwen2.5-Instruct derivative, fully fine-tuned by Sascha Frank for Retrieval-Augmented Generation (RAG) tasks. It focuses on improving context-grounded answering, abstention behavior, and structured output generation, particularly for German-language applications. This model supports a 32,768 token context length and is optimized for research and experimental RAG workflows.
Loading preview...
Overview
This model, tsft-rag-qwen2.5-0.5b-instruct, is a 0.5 billion parameter variant from the TSFT-RAG model family, developed by Sascha Frank AI Research. It is a full-parameter fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct, specifically designed to enhance its performance in Retrieval-Augmented Generation (RAG) systems. The fine-tuning process aimed to improve several critical RAG behaviors, including answering from supplied context, abstaining when context is insufficient, following task-specific output requirements, producing structured JSON, and generating source-aware responses.
Key Capabilities
- Improved Hard-Negative Handling: Significantly better at identifying and rejecting questions not supported by the provided context, with a notable increase from 0% to 29.38% success in hard-negative cases compared to the base model.
- Structured Output: Enhanced ability to produce structured JSON output and adhere to other task-specific output formats.
- Citation-Oriented Responses: Better at generating source-aware and citation-related answers.
- German Language Focus: Primarily trained on a task-specific German-language corpus.
- Long Context: Supports a context length of 32,768 tokens.
Good For
- Context-Grounded Question Answering: Excels in scenarios where answers must be strictly derived from provided evidence.
- Abstention Evaluation: Useful for researching and evaluating model behavior when context does not support an answer.
- Structured RAG Response Generation: Ideal for applications requiring specific output formats, such as JSON.
- Research on Fine-Tuning: Suitable for comparing base and fully fine-tuned small language models in RAG contexts.
- Resource-Constrained RAG: A 0.5B parameter model suitable for local or resource-limited RAG experiments.
It's important to note that while it shows strong gains in specific RAG behaviors, there is a trade-off, with a measured decrease in the grounded-QA score compared to the base model.