yavuz-ai/qwen3-4b-cai-sft

TEXT GENERATIONConcurrent Unit Cost:1Model Size:4BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jun 18, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The yavuz-ai/qwen3-4b-cai-sft model is a 4 billion parameter Qwen3-Instruct variant, fine-tuned using a supervised Constitutional AI (CAI) reproduction method. It was trained on self-critiqued, revised answers to red-team prompts, based on a 5-principle constitution. This model is primarily a research artifact for reproducibility and failure analysis, specifically exploring the tension between harmlessness and over-refusal in AI responses.

Loading preview...

Model Overview

yavuz-ai/qwen3-4b-cai-sft is a 4 billion parameter model based on the Qwen3-Instruct architecture. It represents the supervised learning (SL-CAI) stage of a Constitutional AI (CAI) reproduction, as described by Bai et al. (2022). The model was fine-tuned by generating self-critiqued and revised answers to red-team prompts, adhering to a 5-principle constitution adapted from public sources.

Key Characteristics

  • Constitutional AI Reproduction: Focuses on replicating the supervised stage of Constitutional AI, where the model learns from its own revised outputs.
  • Research Artifact: This model is explicitly published as a research artifact for reproducibility and analysis, particularly concerning the trade-offs between safety and over-refusal.
  • Training Data: Fine-tuned using LoRA SFT on approximately 4,800 self-revisions generated from hh-rlhf harmless-base prompts.
  • Performance Note: During its development stage, evaluations showed an increase in over-refusal on benign prompts (from 0.327 to 0.431) while maintaining safety scores around 1.0. This highlights the tension between harmlessness and over-refusal.

Intended Use

This model is not intended as a general-purpose assistant. Its primary purpose is for:

  • Reproducibility: Allowing researchers to reproduce the supervised stage of Constitutional AI.
  • Failure Analysis: Investigating the dynamics and challenges of Constitutional AI, especially the balance between safety and potential over-refusal. Developers interested in the nuances of AI alignment and safety mechanisms will find this model and its associated research valuable.

Finetunes
1 models