Joint contrastive learning and frequency domain defense against adversarial examples | Neural Computing and Applications Skip to main content
Springer Nature Link
Log in

Joint contrastive learning and frequency domain defense against adversarial examples

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Deep neural networks (DNNs) are vulnerable to being attacked by adversarial examples, leading to DNN misclassification. Perturbations in adversarial examples usually exist in the form of noise. In this paper, we proposed a lightweight joint contrastive learning and frequency domain denoising network (CFNet), which can effectively remove adversarial perturbations from adversarial examples. First, CFNet separates the channels of the features obtained by the multilayer convolution of the adversarial examples, and the separated feature maps are used to calculate the similarity with the high- and low-frequency feature maps obtained by Gaussian low-pass filtering of the clean examples. Second, by adjusting the network’s attention to high-frequency feature images, CFNet can effectively remove the perturbations in adversarial examples and obtain reconstructed examples with high visual quality. Finally, to further improve the robustness of CFNet, contrastive regularization is proposed to bring the reconstructed examples back to the manifold decision boundary of clean examples, thus improving the classification accuracy of reconstructed examples. On the CIFAR-10 dataset, compared with the existing state-of-the-art defense model, the defense accuracy of CFNet is improved by 16.93% and 5.67% under untargeted and targeted projected gradient descent attacks, respectively. The AutoAttack untargeted attack defense accuracy increased by 30.81%. Experiments show that our approach provides better protection than existing state-of-the-art approaches, especially against unseen (untrained) types of attacks and adaptive attacks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Availability of data and materials

For our experiments, we use MNIST, CIFAR-10 and Caltech 101 data, which are publicly available.

Code availability

The code for this study is not publicly available until the paper is published, but is available from the corresponding author Zhi Li on reasonable request.

References

  1. He K, Zhang X, Ren S., Sun J (2016) Deep residual learning for image recognition. In: Conference on computer vision and pattern recognition, pp 770–778

  2. Ren H, Huang T, Yan H (2021) Adversarial examples: attacks and defenses in the physical world. Int J Mach Learn Cybern 12(11):3325–3336

    Article  Google Scholar 

  3. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow IJ, Fergus R (2014) Intriguing properties of neural networks. In: International Conference on Learning Representations

  4. Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. In International conference on learning representations

  5. Xu K, Zhang G, Liu S, Fan Q, Sun M, Chen H, Chen P-Y, Wang Y, Lin X (2020) Adversarial t-shirt! evading person detectors in a physical world. In: European conference on computer vision. Springer, pp 665-681

  6. Eykholt K, Evtimov I, Fernandes E, Li B, Rahmati A, Xiao C, Prakash A, Kohno T, Song D (2018) Robust physical-world attacks on deep learning visual classification. In: Conference on computer vision and pattern recognition

  7. Minervini P, Demeester T, Rocktaschel T, Riedel S (2017) Adversarial sets for regularising neural link predictors. In: Proceedings of the 33rd conference on uncertainty in artificial intelligence (UAI)

  8. Jia R, Liang P (2017) Adversarial examples for evaluating reading comprehension systems. arXiv preprint arXiv:1707.07328

  9. Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2018) Towards deep learning models resistant to adversarial attacks. In: 6th International conference on learning representations

  10. Tramer F, Kurakin A, Papernot N et al. (2018) Ensemble adversarial training: attacks and defenses. In: Proceedings of the international conference on learning representations, pp 1–20

  11. Liao N, Wang S, Xiang L et al (2022) Achieving adversarial robustness via sparsity. Mach Learn 111:685–711

    Article  MathSciNet  MATH  Google Scholar 

  12. Dziugaite GK,Ghahramani Z, Roy DM A study of the effect of jpg compression on adversarial images. arXiv preprint arXiv:1608.00853,2016

  13. Xie C, Wang J, Zhang Z et al. (2018) Mitigating adversarial effects through randomization. In: Proceedings of the international conference on learning representations, pp 1–16

  14. Luo Y, Boix X, Roig G, et al. Foveation-based mechanisms alleviate adversarial examples[C].Proceedings of the International Conference on Learning Representations,2016: 1-25

  15. Samangouei P, Kabkab M, Chellappa R et al. (2018) Defense-GAN: protecting classifiers against adversarial attacks using generative models. arXiv: Computer Vision and Pattern Recognition, pp 3842–3846

  16. Jin G, Shen S, Zhang D et al. (2019) APE-GAN: adversarial perturbation elimination with GAN. In: Proceedings of the IEEE international conference on acoustics, speech and signal processing, pp 3842–3846

  17. Zhou D, Liu T, Han B, Wang N, Peng C, Gao X (2021) Towards defending against adversarial examples via attack-invariant features. In: Proceedings of the38th international conference on machine learning, pp 12835–12845

  18. Echeberria-Barrio X, Gil-Lerchundi A, Egana-Zubia J et al (2022) Understanding deep learning defenses against adversarial examples through visualizations for dynamic risk assessment. Neural Comput Appl. https://doi.org/10.1007/s00521-021-06812-y

    Article  Google Scholar 

  19. Liao F, Liang M,Dong Y et al. (2018) Defense against adversarial attacks using high-level representation guided denoiser. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1778–1787

  20. Kherchouche A, Fezza SA, Hamidouche W (2021) Detect and defense against adversarial examples in deep learning using natural scene statistics and adaptive denoising. Neural Comput Appl. https://doi.org/10.1007/s00521-021-06330-x

    Article  Google Scholar 

  21. Kurakin A,Goodfellow I, Bengio S (2018) Adversarial examples in the physical world. In: Proceedings of the international conference on learning representations workshop, pp 1–14

  22. Madry A, Makelov A, Schmidt L et al. (2018) Towards deep learning models resistant to adversarial attacks. In: Proceedings of the international conference on learning representations, pp 1–27

  23. Dong Y, Liao F, Pang T et al. (2018) Boosting adversarial attacks with momentum. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9185–9193

  24. Papernot N, McDaniel P, Jha S et al. (2016) The limitations of deep learning in adversarial settings. In: Proceedings of the IEEE European symposium on security and privacy, pp 372–387

  25. Carlini N, Wagner D (2017) Towards evaluating the robustness of neural networks. In: Proceedings of the IEEE symposium on security and privacy, pp 39–57

  26. Rony J, Hafemann LG, Oliveira LS, Ayed IB, Sabourin R, Granger E (2019) Decoupling direction and norm for efficient gradient-based L2 adversarial attacks and defenses. In: Conference on computer vision and pattern recognition, pp 4322–4330

  27. Moosavi-Dezfooli SM, Fawzi A, Frossard P (2016) Deepfool: a simple and accurate method to fool deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2574–2582

  28. Croce F, Hein M (2020) Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In: Proceedings of the 37th international conference on machine learning

  29. Sooksatra K, Rivas P (2022) Evaluation of adversarial attacks sensitivity of classifiers with occluded input data. Neural Comput Appl. https://doi.org/10.1007/s00521-022-07387-y

    Article  Google Scholar 

  30. Meng D, Chen H (2017) MagNet: a two-pronged defense against adversarial examples. In: Proceedings of the conference on computer and communications security, pp 135–147

  31. Yin D, Gontijo Lopes R, Shlens J et al. (2019) A fourier perspective on model robustness in computer vision. Adv Neural Inf Proc Syst 32

  32. Zhang Z, Jung C, Liang X (2019) Adversarial defense by suppressing high-frequency components. arXiv preprint arXiv:1908.06566

  33. Olivier R, Raj B, Shah M (2021) High-frequency adversarial defense for speech and audio. In: ICASSP 2021–2021 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2995–2999. https://doi.org/10.1109/ICASSP39728.2021.9414525

  34. Henaff OJ, Srinivas A, De Fauw J, Razavi A, Doersch C, Eslami SM, van den Oord A (2019) Data-efficient image recognition with contrastive predictive coding. arXiv preprint arXiv:1905.09272

  35. Sermanet P, Lynch C, Chebotar Y, Hsu J, Jang E, Schaal S, Levine S, Google Brain (2018) Time-contrastive networks: self-supervised learning from video. In: ICRA, pp 1134–1141

  36. Rim, Daniela N, Heo D, Choi H (2021) Adversarial training with contrastive learning in NLP. arXiv preprint arXiv:2109.09075

  37. Park T, Efros AA, Zhang R et al. (2020) Contrastive learning for unpaired image-to-image translation. In: European conference on computer vision. Springer, Cham, pp 319–345

  38. Ding C, Wang M, Zhou Z, Huang T, Wang X, Li J (2022) Siamese transformer network-based similarity metric learning for cross-source remote sensing image retrieval. Neural Comput. Appl 1–18

  39. Zhou Z et al (2021) Residual visualization-guided explainable copy-relationship learning for image copy detection in social networks. Knowl-Based Syst 228:107287

    Article  Google Scholar 

  40. Zhou Z et al (2022) GAN-siamese network for cross-domain vehicle re-identification in intelligent transport systems. IEEE Trans Netw Sci Eng. https://doi.org/10.1109/TNSE.2022.3199919

    Article  MathSciNet  Google Scholar 

  41. Simonyan, Karen, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  42. Tanay T, Griffin L (2016) A boundary tilting persepective on the phenomenon of adversarial examples. arXiv preprint arXiv:1608.07690

  43. Feinman R, Curtin RR, Shintre S et al. (2017) Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410

  44. Jinyu T, Jiantao Z, Yuanman L, Jia D (2021) Detecting adversarial examples from sensitivity inconsistency of spatial-transform domain. In: AAAI conference on artificial intelligence

  45. Hirschmuller H (2007) Stereo processing by semiglobal matching and mutual information. IEEE Trans Pattern Anal Mach Intell 30(2):328–341

    Article  Google Scholar 

  46. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  47. Jensen MB, Philipsen MP, Møgelmose A, Moeslund TB, Trivedi MM (2016) Vision for looking at traffic lights: issues, survey, and perspectives. IEEE Trans Intell Transp Syst 17(7):1800–1815

    Article  Google Scholar 

  48. Fei-Fei L, Fergus R, Perona P (2004) Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: 2004 Conference on computer vision and pattern recognition workshop, pp 178–178. https://doi.org/10.1109/CVPR.2004.383.

  49. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Conference on computer vision and pattern recognition, pp 770–778

  50. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25

  51. Kurakin A, Goodfellow IJ, Bengio S (2016) Adversarial examples in the physical world. CoRR, abs/1607.02533

  52. Xie C, Wu Y, Maaten LVd, Yuille AL, He K (2019) Feature denoising for improving adversarial robustness. In: CVPR

  53. Shao R, Perera P, Yuen PC, Patel VM (2022) Open-set adversarial defense with clean-adversarial mutual learning. Int J Comput Vis 130(4):1070–87

    Article  Google Scholar 

  54. Guo C, Rana M, Cisse M, van der Maaten L (2018) Countering adversarial images using input transformations. In: ICLR

  55. Prakash A, Moran N, Garber S, DiLillo A, Storer J (2018) Detecting adversarial attacks with pixel detection. In: CVPR 2018

  56. Sun B, Tsai NH, Liu F, Yu R, Su H (2019) Adversarial defense by stratified convolutional sparse coding. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11447–11456

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 62062023, and Guizhou Science and Technology Plan Project under Grant ZK[2021]-YB314, and Stadholder Foundation of Guizhou Province (Grant No. 2007(14))

Author information

Authors and Affiliations

  1. Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, 550025, China

    Jin Yang, Zhi Li, Shuaiwei Liu, Bo Hong & Weidong Wang

Authors
  1. Jin Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Zhi Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Shuaiwei Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Bo Hong
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Weidong Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

JY executed the research. ZL directed the research. WL, BH and WW prepared the datasets.

Corresponding author

Correspondence to Zhi Li.

Ethics declarations

Confict of interest

The authors declare that they have no conflict of interest.

Ethical approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A

Appendix A

On the MNIST test dataset, the network structures of classification \(\text{Model}\_A\), \(\text{Model}\_B\) and \(\text{Model}\_C\) are shown in Table 8:

Table 8 The architecture of models
Full size table

MNIST: In the training phase, we generate a combined dataset from the FGSM and CW attack algorithms, which contains clean examples, adversarial examples and labels. FGSM use an untargeted attack with a perturbation chosen randomly from [0.25, 0.3, 0.35], \(cli{{p}_{\min }}=0\) and \(\text{clip}_\text{max}=1\). The CW attack confidence=10, \(\text{max}\_{\text{iter - ations = 20}}\). The detailed settings of the attack algorithm used in the test phase experiments are as follows:

PGD: We use the \(L_{\infty }\) norm PGD method to craft adversarial examples. The default perturbation budget is set to 0.3. The default number of iterations is set to 40. The attack step size is set to 0.01.

DDN: The number of iterations is set to 100. The factor to modify the norm at each iteration is set to 0.05. The number of quantization levels is set to 256.

CW: We use the \(L_{2}\) norm CW method to craft adversarial examples. The maximum number of iterations is set to 1000. The confidence of the adversarial examples is set to 1. The initial value of the constant is set to 1.

JSMA: The highest percentage of pixels that can be modified is set to 1.0. The perturb length is set to 1.0.

AA: The default perturbation budget is set to 0.3. The default number of iterations is set to 100.

CIFAR-10: In the training phase, two attack algorithms, the PGD targeted attack and untargeted attack, are used to construct the adversarial example dataset, the PGD perturbation size is set to \(\varepsilon =8/255\), and the parameters of the attack algorithm used in the testing phase are set as follows:

PGD: We use the \(L_{\infty }\) norm PGD method to craft adversarial examples. The default perturbation budget is set to 8/255. The default number of iterations is set to 40. The attack step size is set to 0.01.

DDN: The number of iterations is set to 100. The factor to modify the norm at each iteration is set to 0.05. The number of quantization levels is set to 256.

CW: We use the \(L_{2}\) norm CW method to craft adversarial examples. The maximum number of iterations is set to 500. The confidence of the adversarial examples is set to 1. The initial value of the constant is set to 1.

JSMA: The highest percentage of pixels that can be modified is set to 1.0. The perturbation length is set to 1.0.

AA: The default perturbation budget is set to 8/255. The default number of iterations is set to 100.

Caltech 101: The training dataset is generated using the FGSM and LBFGS attack algorithms, with the perturbation budget set as follows:

FGSM: The perturbation budget is set to 8/255, and the default values of \(clip\_min\) and \(clip\_max\) are set to 0 and 1, respectively.

PGD: We use the \({{L}_{\infty }}\) norm PGD method to craft adversarial examples. The default perturbation budget is set to 0.08. The default number of iterations is set to 40. The attack step size is set to 0.01.

DeepFool: The nb_candidate is set to 101, and max_iter is 101.

BIM: The eps is set to 0.04, and the alpha is 1/255.

CW: We use the \({{L}_{2}}\) norm CW method to craft adversarial examples. The maximum number of iterations is set to 500. The confidence of the adversarial examples is set to 1. The initial value of the constant is set to 1.

The visualization results of the MNIST dataset under the PGD and FGSM attack algorithms with the perturbation budget set to 0.3. As shown in Fig. 14, the first row represents the adversarial examples, and the second row represents the reconstructed clean examples.

Fig. 14
figure 14

The visualization results of the MNIST dataset under various attack

Full size image

Visualization results on the CIFAR-10 dataset under different attack algorithms are shown in Fig. 15. The PGD and FGSM perturbation budgets are set to 0.06 and 0.07, respectively; the first row represents the adversarial examples, and the second row represents the reconstructed examples.

Fig. 15
figure 15

The visualization results of the CIFAR-10 dataset under various attack

Full size image

The original Caltech 101 image and the reconstructed result are shown in Figs. 16 and 17. The first row indicates the original image, and the second row indicates the reconstructed image.

Fig. 16
figure 16

(1) Caltech 101

Full size image
Fig. 17
figure 17

(2) Caltech 101

Full size image

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, J., Li, Z., Liu, S. et al. Joint contrastive learning and frequency domain defense against adversarial examples. Neural Comput & Applic 35, 18623–18639 (2023). https://doi.org/10.1007/s00521-023-08688-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08688-6

Keywords