Breaking Free How To Hack Safety Guardrails In Blackbox Diffusion Models

# Breaking Free: How to Hack Safety Guardrails in Black-Box Diffusion Models!

**arXiv ID:** 2402.04699
**Authors:** Shashank Kotyan, Po-Yuan Mao, Pin-Yu Chen, Danilo Vasconcellos Vargas
**Published:** 2024-02-07T09:39:29Z
**Abstract:**
Deep neural networks can be exploited using natural adversarial samples, which do not impact human perception. Current approaches often rely on deep neural networks' white-box nature to generate these adversarial samples or synthetically alter the distribution of adversarial samples compared to the training distribution. In contrast, we propose EvoSeed, a novel evolutionary strategy-based algorithmic framework for generating photo-realistic natural adversarial samples. Our EvoSeed framework uses auxiliary Conditional Diffusion and Classifier models to operate in a black-box setting. We employ CMA-ES to optimize the search for an initial seed vector, which, when processed by the Conditional Diffusion Model, results in the natural adversarial sample misclassified by the Classifier Model. Experiments show that generated adversarial images are of high image quality, raising concerns about generating harmful content bypassing safety classifiers. Our research opens new avenues to understanding the limitations of current safety mechanisms and the risk of plausible attacks against classifier systems using image generation. Project Website can be accessed at: https://shashankkotyan.github.io/EvoSeed.

## Skill Description

This skill is generated from the arXiv paper: Breaking Free: How to Hack Safety Guardrails in Black-Box Diffusion Models! (2402.04699).

## How to Use

[To be filled in by the user or by future automation]

## References

- [arXiv:2402.04699](http://arxiv.org/abs/2402.04699v2)