arXiv:2311.14029cs.CV
Understanding the Vulnerability of CLIP to Image Compression
Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.
Abstract
CLIP is a widely used foundational vision-language model that is used for zero-shot image recognition and other image-text alignment tasks. We demonstrate that CLIP is vulnerable to change in image quality under compression. This surprising result is further analysed using an attribution method-Integrated Gradients. Using this attribution method, we are able to better understand both quantitatively and qualitatively exactly the nature in which the compression affects the zero-shot recognition accuracy of this model. We evaluate this extensively on CIFAR-10 and STL-10. Our work provides the basis to understand this vulnerability of CLIP and can help us develop more effective methods to improve the robustness of CLIP and other vision-language models.
Kind of an odd blog post -- is this AI generated? Or maybe written by someone who isn't a native English speaker? The writing feels off. In any case, maybe worth linking directly to the paper, it's an easy read and more informative: https://arxiv.org/abs/2311.14029 The tl;dr is…
heyitsguayView on HNParent context