Overview
In computer vision, deep learning has become a dominant approach for a broad range of applications, from autonomous driving to surveillance and security. Although deep neural networks have achieved remarkable success in solving complex tasks, research has shown that they are vulnerable to adversarial attacks. Such attacks typically involve adding subtle perturbations to the input, causing the neural network to produce incorrect outputs. For images, these perturbations are often too minor to be perceived by the human eye, yet they can completely deceive neural network models and therefore create potential risks in real-world applications. As an important branch of computer vision, content-based image retrieval (CBIR) systems are also exposed to the threat of adversarial attacks.
The Adaptive Targeted Attack Generative Adversarial Network (ATA-GAN) is an adversarial attack method for image retrieval. It uses an asymmetric Feature Integration Module (FIM) to enhance the recognition and fusion of features from the target image and the original image, enabling targeted adversarial attacks on retrieval systems.
Key Contributions
1. For content-based image retrieval, ATA-GAN proposes an adaptive method for generating targeted adversarial examples and is designed to attack multiple specified targets.
2. ATA-GAN introduces an attention-based Feature Integration Module that selects discriminative features from both the original image and the target image to generate perturbations.
3. ATA-GAN demonstrates successful attack performance across multiple datasets, with the similarity between the adversarial examples and the target image rank lists exceeding 91.2%.
Network Architecture
Figure 1. Network architecture of ATA-GAN
As shown in Figure 1, the encoder-decoder structures of the generator and discriminator in ATA-GAN are similar to those used in the pix2pix model. However, unlike pix2pix, this approach extends the GAN framework into a conditional GAN (CGAN) and uses the target image as an auxiliary input. On this basis, a network-level feature integration module is designed to select discriminative features from the target image and the original image.
Unlike image classification tasks, image retrieval datasets do not provide class labels. Therefore, ATA-GAN adopts the global descriptor of the target image to assist in training the generator.
Feature Integration Module
Figure 2. Feature Integration Module
The Feature Integration Module (FIM) is designed to focus more strongly on the features of the target image while suppressing or ignoring features of the original image. The module adopts an asymmetric structure to ensure that the target image serves both as the conditional input to the network and as the basis for generating perturbations, as illustrated in Figure 2.
Experimental Evaluation and Results
Overall, the model can generate targeted adversarial examples for any given pair of target and original images. Unlike single-target attack methods, which require the model to be retrained whenever the target changes, ATA-GAN needs to be trained only once and can then attack multiple targets. In addition, ATA-GAN introduces the Feature Integration Module to explore the relationship between the original image and the target image.
The experiments were conducted by training the model on the SfM120k dataset and testing it on the Oxford5k and Paris6k datasets. The results demonstrate the effectiveness of the proposed attack method.
Figure 3. Adversarial attack results of ATA-GAN
Figure 3 presents representative adversarial attack results. The first row shows the top 10 images in the target image rank list on Oxford5k. The second row shows the top 10 retrieval results for the adversarial example generated from the original image and the target image. The final row shows the top 10 retrieval results for the adversarial example generated from random noise and the target image. The images marked with red boxes are not included in the target image’s top 10 retrieval results.
Keywords
Image Retrieval; Content-Based Image Retrieval; Adversarial Attack; Generative Adversarial Network; ATA-GAN; Feature Integration Module
