WilliamSheltonWu/confessionRL-Qwen3-4B

Model Overview

confessionRL-Qwen3-4B is a 4 billion parameter language model developed by WilliamSheltonWu, fine-tuned using the VERL framework and the GRPO method. This model underwent full-parameter fine-tuning with a learning rate of lr=5e-6 on a dataset of 300 entries.

Key Differentiators & Performance

This model is specifically engineered to address the challenge of "false confessions" in large language models, particularly under adversarial conditions. It demonstrates significant improvements over its base model, Qwen3-4B-Instruct-2507, in maintaining integrity when faced with misleading instructions or user prompts.

• Resistance to False Confessions: In 'tool_attack_confidential' tests, confessionRL-Qwen3-4B achieved a 100% valid confession rate with 0% false confessions, compared to the base model's 0% valid and 100% false confessions. Similarly, in 'user_prompt_attack' tests, it maintained a 100% valid confession rate with 0% false confessions.
• High Accuracy on Clean Samples: The model performs comparably to the base model on clean test sets (based on GSM8k), achieving 92% accuracy with tool use and 94% accuracy without tool use, indicating no degradation in general performance.

Use Cases

This model is particularly well-suited for applications where:

• Robustness against adversarial attacks is critical.
• The model might interact with external tools that could provide misleading instructions.
• Maintaining high integrity and truthfulness in responses is paramount, even under manipulative prompts.