ibm-granite/granite-3.1-8b-lora-intrinsics-v0.1
Granite 3.1 8B Instruct - Intrinsics LoRA v0.1 is an 8 billion parameter LoRA adapter developed by IBM Research for the ibm-granite/granite-3.1-8b-instruct model, featuring a 32768 token context length. This experimental adapter provides access to Uncertainty Quantification, Hallucination Detection, and Safety Exception intrinsics, allowing the base model to output certainty scores, detect hallucinations in RAG responses, and identify unsafe user queries. It is designed for community testing and feedback on these novel intrinsic functionalities.
Loading preview...
Overview
Granite 3.1 8B Instruct - Intrinsics LoRA v0.1 is an experimental LoRA adapter developed by IBM Research for the 8 billion parameter ibm-granite/granite-3.1-8b-instruct model. It extends the base model's capabilities by integrating three key "intrinsics": Uncertainty Quantification, Hallucination Detection, and Safety Exception. This model is released for community preview and feedback, not for production deployment.
Key Capabilities
- Uncertainty Intrinsic: Provides a calibrated certainty score (0-9) for model responses, indicating confidence levels (5% to 95%).
- Hallucination Detection (RAG) Intrinsic: Detects when an assistant's response, given supporting documents, is not supported by those documents, outputting 'Y' for hallucination and 'N' otherwise.
- Safety Exception Intrinsic: Acts as a binary classifier to identify unsafe user queries, responding with 'Y' for unsafe and 'N' for safe, covering categories like violence, threats, and explicit content.
- Retains Base Model Abilities: The LoRA adapter is lightly tuned to preserve the full abilities of the original Granite 3.1 8B Instruct model.
Performance and Training
Evaluations show that the intrinsics' performance is not degraded compared to separate intrinsic models, with low percent error for Hallucination Detection and Safety Exception, and low Mean Absolute Error for Uncertainty. RAG performance, measured by RAGAS faithfulness and correction metrics, also remains comparable to the base model. The model was fine-tuned using various datasets for each intrinsic, including BigBench, MRQA, MultiDoc2Dial, and Anthropic/hh-rlhf.
Usage
Each intrinsic is invoked via a specific generation role (e.g., certainty, hallucination, safety) and responds with a single token. Examples are provided for integrating these intrinsics into RAG workflows using frameworks like Prompt Declaration Language (PDL) and SGLang, demonstrating how to combine multiple intrinsic checks in a multi-turn conversation.