ibm-granite/granite-3.0-8b-lora-intrinsics-v0.1

Model Overview

ibm-granite/granite-3.0-8b-lora-intrinsics-v0.1 is an experimental 8 billion parameter LoRA adapter developed by IBM Research. It extends the capabilities of the ibm-granite/granite-3.0-8b-instruct model by integrating three key intrinsics: Uncertainty, Hallucination Detection, and Safety Exception. This model is released for community testing and feedback, not for production deployments.

Key Capabilities

• Uncertainty Intrinsic: Provides a calibrated certainty score (0-9) for model responses, indicating confidence levels.
• Hallucination Detection (RAG) Intrinsic: Identifies when an assistant's response, supported by documents, is not factually grounded in those documents.
• Safety Exception Intrinsic: Acts as a binary classifier to detect unsafe user queries, covering harms like violence, threats, and explicit content.
• Retains Base Model Abilities: The LoRA finetune ensures the model largely preserves the original capabilities of the Granite 3.0 8B Instruct model.

Usage and Evaluation

Each intrinsic can be invoked using specific generation roles within the chat template. The model supports combining intrinsics in multi-turn conversations, with examples provided using Prompt Declaration Language (PDL) and SGLang. Evaluations show that the intrinsic performance is not degraded compared to separate models, and RAG performance remains consistent with the base Granite 3.0 8B Instruct model.

Training Details

The LoRA adapter was finetuned using various public datasets for each intrinsic:

• Uncertainty: Datasets like BigBench, MRQA, Trivia QA, and others, with certainty scores obtained via the Thermometer method.
• Hallucination Detection: MultiDoc2Dial and QuAC, using techniques from Achintalwar et al. for label creation.
• Safety Exception: Datasets including yahma/alpaca-cleaned, nvidia/Aegis-AI-Content-Safety-Dataset-1.0, and a subset of Anthropic/hh-rlhf.