arXiv:2210.17316eess.AS
There is more than one kind of robustness: Fooling Whisper with adversarial examples
Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.
Abstract
Whisper is a recent Automatic Speech Recognition (ASR) model displaying impressive robustness to both out-of-distribution inputs and random noise. In this work, we show that this robustness does not carry over to adversarial noise. We show that we can degrade Whisper performance dramatically, or even transcribe a target sentence of our choice, by generating very small input perturbations with Signal Noise Ratio of 35-45dB. We also show that by fooling the Whisper language detector we can very easily degrade the performance of multilingual models. These vulnerabilities of a widely popular open-source model have practical security implications and emphasize the need for adversarially robust ASR.
Yeah, there have been several papers with attacks on Whisper: - Inject adversarial noise to make it transcribe what you want (https://arxiv.org/abs/2210.17316) - Stop it from transcribing (https://arxiv.org/abs/2405.06134) - Adversarial prompt injection to make it translate…
woodsonView on HNParent context