Abstract
White box adversarial perturbations are generated via iterative optimization algorithms most often by minimizing an adversarial loss on a \(\ell _p\) neighborhood of the original image, the so-called distortion set. Constraining the adversarial search with different norms results in disparately structured adversarial examples. Here we explore several distortion sets with structure-enhancing algorithms. These new structures for adversarial examples might provide challenges for provable and empirical robust mechanisms. Because adversarial robustness is still an empirical field, defense mechanisms should also reasonably be evaluated against differently structured attacks. Besides, these structured adversarial perturbations may allow for larger distortions size than their \(\ell _p\) counterpart while remaining imperceptible or perceptible as natural distortions of the image. We will demonstrate in this work that the proposed structured adversarial examples can significantly bring down the classification accuracy of adversarially trained classifiers while showing a low \(\ell _2\) distortion rate. For instance, on ImagNet dataset the structured attacks drop the accuracy of the adversarial model to near zero with only 50% of \(\ell _2\) distortion generated using white-box attacks like PGD. As a byproduct, our findings on structured adversarial examples can be used for adversarial regularization of models to make models more robust or improve their generalization performance on datasets that are structurally different.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Allen-Zhu, Z., Hazan, E., & Hu, W., et al. (2017). Linear convergence of a frank-wolfe type algorithm over trace-norm balls. In Advances in neural information processing systems (pp. 6191–6200).
Athalye, A., Carlini, N., & Wagner, D. (2018). Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. arXiv:1802.00420
Candès, E. J., & Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational mathematics, 9(6), 717.
Carlini, N., & Wagner, D. (2017). Towards evaluating the robustness of neural networks. In 2017 IEEE symposium on security and privacy (SP) (pp. 39–57). IEEE.
Carlini, N., Athalye, A., & Papernot, N., et al. (2019). On evaluating adversarial robustness. arXiv:1902.06705
Chen, J., Yi, J., & Gu, Q. (2018). A frank-wolfe framework for efficient and effective adversarial attacks. arXiv:1811.10828
Cheung, E., & Li, Y. (2017). Projection free rank-drop steps. arXiv:1704.04285
Cohen, J. M., Rosenfeld, E., & Kolter, J. Z. (2019). Certified adversarial robustness via randomized smoothing. arXiv:1902.02918
Croce, F., & Hein, M. (2019). Sparse and imperceivable adversarial attacks. In Proceedings of the IEEE international conference on computer vision (pp. 4724–4732).
Croce, F., & Hein, M. (2020). Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, PMLR (pp. 2206–2216).
Cui, J., Liu, S., & Wang, L., et al. (2021). Learnable boundary guided adversarial training. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 15721–15730).
Demyanov, V. F., & Rubinov, A. M. (1970). Approximate methods in optimization problems. In Modern analytic and computational methods in science and mathematics.
Dudik, M., Harchaoui, Z., & Malick, J. (2012). Lifted coordinate descent for learning with trace-norm regularization. In Artificial intelligence and statistics (pp. 327–336).
Dunn, J. C. (1979). Rates of convergence for conditional gradient algorithms near singular and nonsingular extremals. SIAM Journal on Control and Optimization, 17(2), 187–211.
Engstrom, L., Tran, B., & Tsipras, D., et al. (2017). A rotation and a translation suffice: Fooling cnns with simple transformations. arXiv:1712.02779
Fazel, M., Hindi, H., & Boyd, S. P. (2001). A rank minimization heuristic with application to minimum order system approximation. In Proceedings of the 2001 American control conference (Cat. No. 01CH37148) (pp. 4734–4739). IEEE.
Frank, M., & Wolfe, P. (1956). An algorithm for quadratic programming. Naval Research Logistics Quarterly, 3(1–2), 95–110.
Freund, R. M., Grigas, P., & Mazumder, R. (2017). An extended frank-wolfe method with “in-face’’ directions, and its application to low-rank matrix completion. SIAM Journal on Optimization, 27(1), 319–346.
Garber, D., & Hazan, E. (2013a). A linearly convergent conditional gradient algorithm with applications to online and stochastic optimization. arXiv:1301.4666
Garber, D., & Hazan, E. (2013b). Playing non-linear games with linear oracles. In 2013 IEEE 54th annual symposium on foundations of computer science (pp. 420–428). IEEE.
Garber, D., & Hazan, E. (2015). Faster rates for the frank-wolfe method over strongly-convex sets. In 32nd International conference on machine learning, ICML 2015.
Garber, D., Sabach, S., & Kaplan, A. (2018). Fast generalized conditional gradient method with applications to matrix recovery problems. arXiv:1802.05581
Gatys, L. A., Ecker, A. S., & Bethge, M., et al. (2017). Controlling perceptual factors in neural style transfer. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3985–3993).
Gilmer, J., Adams, R.P., & Goodfellow, I., et al. (2018). Motivating the rules of the game for adversarial example research. arXiv:1807.06732
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. In International conference on learning representations.
Gragnaniello, D., Marra, F., Verdoliva, L., et al. (2021). Perceptual quality-preserving black-box attack against deep learning image classifiers. Pattern Recognition Letters, 147, 142–149.
Guélat, J., & Marcotte, P. (1986). Some comments on Wolfe’s ‘away step’. Mathematical Programming.
Guo, C., Frank, J. S., & Weinberger, K. Q. (2018). Low frequency adversarial perturbation. arXiv:1809.08758
Guo, Q., Juefei-Xu, F., & Xie, X., et al. (2020). Watch out! motion is blurring the vision of your deep neural networks. arXiv:2002.03500
Hameed, M. Z., & Gyorgy, A. (2021). Perceptually constrained adversarial attacks. arXiv:2102.07140
Harchaoui, Z., Douze, M., & Paulin, M., et al. (2012). Large-scale image classification with trace-norm regularization. In 2012 IEEE conference on computer vision and pattern recognition (pp. 3386–3393). IEEE.
Jaggi, M. (2013). Revisiting frank-wolfe: Projection-free sparse convex optimization. In Proceedings of the 30th international conference on machine learning, CONF (pp. 427–435).
Jaggi, M., & Sulovskỳ, M. (2010). A simple algorithm for nuclear norm regularized problems. In ICML.
Kerdreux, T., & d’Aspremont, A. (2020). Frank-wolfe on uniformly convex sets. arXiv:2004.11053
Kerdreux, T., Pedregosa, F., & d’Aspremont, A. (2018). Frank-wolfe with subsampling oracle. arXiv:1803.07348
Keskar, N. S., Mudigere, D., & Nocedal, J., et al. (2016). On large-batch training for deep learning: Generalization gap and sharp minima. arXiv:1609.04836
Kurakin, A., Goodfellow, I., & Bengio, S. (2016). Adversarial examples in the physical world. arXiv:1607.02533
Lacoste-Julien, S., & Jaggi, M. (2013). An affine invariant linear convergence analysis for frank-wolfe algorithms. arXiv:1312.7864
Lacoste-Julien, S., & Jaggi, M. (2015). On the global linear convergence of Frank–Wolfe optimization variants. In Cortes, C., Lawrence, N. D., Lee, D. D., et al (Eds.). Advances in neural information processing systems (Vol. 28, pp. 496–504). Curran Associates, Inc.
Langeberg, P., Balda, E. R., & Behboodi, A., et al. (2019). On the effect of low-rank weights on adversarial robustness of neural networks. arXiv:1901.10371
Lee, J.D., Recht, B., & Srebro, N., et al. (2010). Practical large-scale optimization for max-norm regularization. In Advances in neural information processing systems (pp. 1297–1305).
Levitin, E. S., & Polyak, B. T. (1966). Constrained minimization methods. USSR Computational Mathematics and Mathematical Physics, 6(5), 1–50.
Liu, H. T. D., Tao, M., & Li, C. L., et al. (2018). Beyond pixel norm-balls: Parametric adversaries using an analytically differentiable renderer. In International conference on learning representations.
Lu, M., Zhao, H., & Yao, A., et al. (2017). Decoder network over lightweight reconstructed feature for fast semantic style transfer. In Proceedings of the IEEE international conference on computer vision (pp. 2469–2477).
Luo, B., Liu, Y., & Wei, L., et al. (2018). Towards imperceptible and robust adversarial example attacks against neural networks. In Thirty-second AAAI conference on artificial intelligence.
Madry, A., Makelov, A., & Schmidt, L., et al. (2017). Towards deep learning models resistant to adversarial attacks. arXiv:1706.06083
Papernot, N., McDaniel, P., & Jha, S., et al. (2016). The limitations of deep learning in adversarial settings. In 2016 IEEE European symposium on security and privacy (EuroS &P) (pp. 372–387). IEEE.
Raghunathan, A., Steinhardt, J., & Liang, P. (2018). Certified defenses against adversarial examples. arXiv:1801.09344
Rauber, J., Brendel, W., & Bethge, M. (2017). Foolbox: A python toolbox to benchmark the robustness of machine learning models. In Reliable machine learning in the wild workshop, 34th international conference on machine learning. http://arxiv.org/abs/1707.04131
Reed, S.E., Akata, Z., & Mohan, S., et al. (2016). Learning what and where to draw. In: Advances in neural information processing systems (pp. 217–225).
Risser, E., Wilmot, P., & Barnes, C. (2017). Stable and controllable neural texture synthesis and style transfer using histogram losses. arXiv:1701.08893
Schmidt, L., Santurkar, S., & Tsipras, D., et al. (2018). Adversarially robust generalization requires more data. In Advances in neural information processing systems (pp. 5014–5026).
Sen, A., Zhu, X., & Marshall, L., et al. (2019). Should adversarial attacks use pixel p-norm? arXiv:1906.02439
Shalev-Shwartz, S., Gonen, A., & Shamir, O. (2011). Large-scale convex minimization with a low-rank constraint. arXiv:1106.1622
Sharif, M., Bauer, L., & Reiter, M. K. (2018). On the suitability of lp-norms for creating and preventing adversarial examples. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 1605–1613).
Stutz, D., Hein, M., & Schiele, B. (2019). Disentangling adversarial robustness and generalization. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 6976–6987)
Tomioka, R., & Suzuki, T. (2013). Convex tensor decomposition via structured schatten norm regularization. In Advances in neural information processing systems (pp. 1331–1339).
Wang, Z., Bovik, A. C., Sheikh, H. R., et al. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.
Wong, E., & Kolter, J. Z. (2017). Provable defenses against adversarial examples via the convex outer adversarial polytope. arXiv:1711.00851
Wong, E., & Kolter, J. Z. (2020). Learning perturbation sets for robust machine learning. arXiv:2007.08450
Wong, E., Schmidt, F. R., & Kolter, J. Z. (2019). Wasserstein adversarial examples via projected sinkhorn iterations. arXiv:1902.07906
Wu, D., Xia, S. T., & Wang, Y. (2020). Adversarial weight perturbation helps robust generalization. arXiv:2004.05884
Xu, K., Liu, S., & Zhao, P., et al. (2018). Structured adversarial attack: Towards general implementation and better interpretability. arXiv:1808.01664
Yan, Z., Guo, Y., & Zhang, C. (2019). Subspace attack: Exploiting promising subspaces for query-efficient black-box attacks. arXiv:1906.04392
Yang, G., Duan, T., & Hu, E., et al. (2020). Randomized smoothing of all shapes and sizes. arXiv:2002.08118
Yang, Y., Zhang, G., & Katabi, D., et al. (2019). Me-net: Towards effective adversarial robustness with matrix estimation. arXiv:1905.11971
Zhang, H., Chen, H., & Xiao, C., et al. (2019). Towards stable and efficient training of verifiably robust neural networks. arXiv:1906.06316
Zhou, B., Khosla, A., & Lapedriza. A., et al. (2016). Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2921–2929).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Liwei Wang.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kazemi, E., Kerdreux, T. & Wang, L. Minimally Distorted Structured Adversarial Attacks. Int J Comput Vis 131, 160–176 (2023). https://doi.org/10.1007/s11263-022-01701-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-022-01701-w