Abstract
Deep generative models (DGMs) have empowered unprecedented innovations in many application domains. However, their security has not been thoroughly assessed when deploying such models in practice, especially in those mission-critical tasks like autonomous driving. In this work, we draw attention to a new attack surface of DGMs, which is the data used in the training phase. We demonstrate that the training data poisoning, the injection of specially-crafted data, are able to teach Trojan behaviors to a DGM without influencing the original training goal. Such Trojan attack will be activated after model deployment only if certain rare triggers are present in an input. For example, a rain-removal DGM after poisoning can, while removing raindrops in input images, change a traffic light from red to green if this traffic light has a specific appearance (i.e. a trigger). Clearly severe consequences can occur if such poisoned model is deployed on vehicle. Our study shows that launching our Trojan attack is feasible on different DGM categories designed for the autonomous driving scenario, and existing defense methods cannot effectively defeat it. We also introduce a concealing technique to make our data poisoning more inconspicuous during the training. In the end, we propose some potential defense strategies inspiring future explorations.
S. Ding and Y. Tian—Both authors contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Azure cognitive services. https://azure.microsoft.com/en-us/services/cognitive-services/
Bloomberg news. https://www.bloomberg.com/news/articles/2018-09-17/self-driving-cars-still-can-t-handle-bad-weather
CNN classifier. https://github.com/srini-ry/ros-traffic-light-classifierr
The intelligence advanced research projects activity. https://www.iarpa.gov/index.php?option=com_content&view=article&id=1150&Itemid=448
Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018)
Chen, B., et al.: Detecting backdoor attacks on deep neural networks by activation clustering. arXiv preprint arXiv:1811.03728 (2018)
Chen, X., Liu, C., Li, B., Lu, K., Song, D.: Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526 (2017)
Fan, Z., Wu, H., Fu, X., Huang, Y., Ding, X.: Residual-guide network for single image deraining. In: Proceedings of the 26th ACM International Conference on Multimedia, MM 2018, pp. 1751–1759. ACM, New York (2018)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Gu, T., Dolan-Gavitt, B., Garg, S.: BadNets: identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733 (2017)
Hayes, J., Melis, L., Danezis, G., De Cristofaro, E.: LOGAN: membership inference attacks against generative models. Proc. Priv. Enhancing Technol. 2019(1), 133–152 (2019)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Ji, Y., Zhang, X., Ji, S., Luo, X., Wang, T.: Model-reuse attacks on deep learning systems. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 349–363. ACM (2018)
Kastner, S., Ungerleider, L.G.: Mechanisms of visual attention in the human cortex. Ann. Rev. Neurosci. 23, 315–341 (2000)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Klein, G., Kim, Y., Deng, Y., Senellart, J., Rush, A.M.: OpenNMT: open-source toolkit for neural machine translation. In: Proceedings of ACL (2017)
Kos, J., Fischer, I., Song, D.: Adversarial examples for generative models. In: 2018 IEEE Security and Privacy Workshops (SPW), pp. 36–42. IEEE (2018)
Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D., Matas, J.: DeblurGAN: blind motion deblurring using conditional adversarial networks. In: Proceedings of CVPR (2018)
Larsson, F., Felsberg, M., Forssen, P.E.: Correlating Fourier descriptors of local patches for road sign recognition. IET Comput. Vis. 5(4), 244–254 (2011)
Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: AOD-NET: all-in-one dehazing network. In: Proceedings of CVPR (2017)
Liu, K., Dolan-Gavitt, B., Garg, S.: Fine-pruning: defending against backdooring attacks on deep neural networks. In: Bailey, M., Holz, T., Stamatogiannakis, M., Ioannidis, S. (eds.) RAID 2018. LNCS, vol. 11050, pp. 273–294. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00470-5_13
Liu, Y., et al.: Trojaning attack on neural networks (2017)
van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Mogelmose, A., Trivedi, M.M., Moeslund, T.B.: Vision-based traffic sign detection and analysis for intelligent driver assistance systems: perspectives and survey. IEEE Trans. Intell. Transp. Syst. 13(4), 1484–1497 (2012)
Pan, J., et al.: SalGAN: visual saliency prediction with generative adversarial networks. arXiv, January 2017
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of ACL (2002)
Pasquini, D., Mingione, M., Bernaschi, M.: Out-domain examples for generative models (2019)
Qian, R., Tan, R.T., Yang, W., Su, J., Liu, J.: Attentive generative adversarial network for raindrop removal from a single image. In: Proceedings of CVPR (2018)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)
Tran, B., Li, J., Madry, A.: Spectral signatures in backdoor attacks. In: Advances in Neural Information Processing Systems, pp. 8011–8021 (2018)
Uricár, M., Krízek, P., Hurych, D., Sobh, I., Yogamani, S., Denny, P.: Yes, we GAN: applying adversarial techniques for autonomous driving. CoRR abs/1902.03442 (2019). http://arxiv.org/abs/1902.03442
Wang, Y., et al.: Tacotron: towards end-to-end speech synthesis. arXiv preprint arXiv:1703.10135 (2017)
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 294–310. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_18
Acknowledgments
The authors would like to thank our paper shepherd Dr. Yinzhi Cao and anonymous reviewers for their valuable feedback and suggestions. This work was supported in part by NSFC-61872180, Jiangsu “Shuang-Chuang” Program, Jiangsu “Six-Talent-Peaks” Program, Ant Financial through the Ant Financial Science Funds for Security Research, US NSF-1816399, NSFC-61425024, and NSFC-61872176.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Ding, S., Tian, Y., Xu, F., Li, Q., Zhong, S. (2019). Trojan Attack on Deep Generative Models in Autonomous Driving. In: Chen, S., Choo, KK., Fu, X., Lou, W., Mohaisen, A. (eds) Security and Privacy in Communication Networks. SecureComm 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 304. Springer, Cham. https://doi.org/10.1007/978-3-030-37228-6_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-37228-6_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-37227-9
Online ISBN: 978-3-030-37228-6
eBook Packages: Computer ScienceComputer Science (R0)