{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,7,21]],"date-time":"2024-07-21T05:14:15Z","timestamp":1721538855930},"reference-count":47,"publisher":"Privacy Enhancing Technologies Symposium Advisory Board","issue":"1","license":[{"start":{"date-parts":[[2021,11,20]],"date-time":"2021-11-20T00:00:00Z","timestamp":1637366400000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/3.0"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,1,1]]},"abstract":"Abstract<\/jats:title>\n Facial recognition tools are becoming exceptionally accurate in identifying people from images. However, this comes at the cost of privacy for users of online services with photo management (e.g. social media platforms). Particularly troubling is the ability to leverage unsupervised learning to recognize faces even when the user has not labeled their images. In this paper we propose Ulixes, a strategy to generate visually non-invasive facial noise masks that yield adversarial examples, preventing the formation of identifiable user clusters in the embedding space of facial encoders. This is applicable even when a user is unmasked and labeled images are available online. We demonstrate the effectiveness of Ulixes by showing that various classification and clustering methods cannot reliably label the adversarial examples we generate. We also study the effects of Ulixes in various black-box settings and compare it to the current state of the art in adversarial machine learning. Finally, we challenge the effectiveness of Ulixes against adversarially trained models and show that it is robust to countermeasures.<\/jats:p>","DOI":"10.2478\/popets-2022-0008","type":"journal-article","created":{"date-parts":[[2021,11,21]],"date-time":"2021-11-21T02:42:43Z","timestamp":1637462563000},"page":"148-165","source":"Crossref","is-referenced-by-count":3,"title":["Ulixes: Facial Recognition Privacy with Adversarial Machine Learning"],"prefix":"10.56553","volume":"2022","author":[{"given":"Thomas","family":"Cilloni","sequence":"first","affiliation":[{"name":"University of Mississippi"}]},{"given":"Wei","family":"Wang","sequence":"additional","affiliation":[{"name":"Xi\u2019an Jiaotong-Liverpool University"}]},{"given":"Charles","family":"Walter","sequence":"additional","affiliation":[{"name":"University of Mississippi"}]},{"given":"Charles","family":"Fleming","sequence":"additional","affiliation":[{"name":"University of Mississippi"}]}],"member":"35752","published-online":{"date-parts":[[2021,11,20]]},"reference":[{"key":"2022062314372015302_j_popets-2022-0008_ref_001","unstructured":"[1] European Parliament and Council of European Union, \u201cRegulation (eu) 2016\/679,\u201d 2016, http:\/\/data.europa.eu\/eli\/reg\/2016\/679\/oj."},{"key":"2022062314372015302_j_popets-2022-0008_ref_002","unstructured":"[2] \u201cAre Organizations Ready for New Privacy Regulations?\u201d [Online]. Available: https:\/\/www.internetsociety.org\/resources\/ota\/2019\/are-organizations-ready-for-new-privacy-regulations\/"},{"key":"2022062314372015302_j_popets-2022-0008_ref_003","doi-asserted-by":"crossref","unstructured":"[3] J. P. Pesce, D. L. Casas, G. Rauber, and V. Almeida, \u201cPrivacy attacks in social media using photo tagging networks: a case study with Facebook,\u201d in Proceedings of the 1st Workshop on Privacy and Security in Online Social Media - PSOSM \u201912. Lyon, France: ACM Press, 2012, pp. 1\u20138. [Online]. Available: http:\/\/dl.acm.org\/citation.cfm?doid=2185354.218535810.1145\/2185354.2185358","DOI":"10.1145\/2185354.2185358"},{"key":"2022062314372015302_j_popets-2022-0008_ref_004","unstructured":"[4] S. Shan, E. Wenger, J. Zhang, H. Li, H. Zheng, and B. Y. Zhao, \u201cFawkes: Protecting privacy against unauthorized deep learning models,\u201d in 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, Aug. 2020, pp. 1589\u20131604. [Online]. Available: https:\/\/www.usenix.org\/conference\/usenixsecurity20\/presentation\/shan"},{"key":"2022062314372015302_j_popets-2022-0008_ref_005","doi-asserted-by":"crossref","unstructured":"[5] C. Gao, V. Chandrasekaran, K. Fawaz, and S. Jha, \u201cFace-off: Adversarial face obfuscation,\u201d 2020.10.2478\/popets-2021-0032","DOI":"10.2478\/popets-2021-0032"},{"key":"2022062314372015302_j_popets-2022-0008_ref_006","doi-asserted-by":"crossref","unstructured":"[6] F. Schroff, D. Kalenichenko, and J. Philbin, \u201cFaceNet: A unified embedding for face recognition and clustering,\u201d in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, Jun. 2015, pp. 815\u2013823. [Online]. Available: http:\/\/ieeexplore.ieee.org\/document\/7298682\/10.1109\/CVPR.2015.7298682","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"2022062314372015302_j_popets-2022-0008_ref_007","doi-asserted-by":"crossref","unstructured":"[7] Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, \u201cDeepFace: Closing the Gap to Human-Level Performance in Face Verification,\u201d in 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, Jun. 2014, pp. 1701\u20131708. [Online]. Available: http:\/\/ieeexplore.ieee.org\/lpdocs\/epic03\/wrapper.htm?arnumber=690961610.1109\/CVPR.2014.220","DOI":"10.1109\/CVPR.2014.220"},{"key":"2022062314372015302_j_popets-2022-0008_ref_008","unstructured":"[8] \u2014\u2014, \u201cWeb-scale training for face identification,\u201d in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, Jun. 2015, pp. 2746\u20132754. [Online]. Available: http:\/\/ieeexplore.ieee.org\/document\/7298891\/"},{"key":"2022062314372015302_j_popets-2022-0008_ref_009","unstructured":"[9] Y. Sun, Y. Chen, X. Wang, and X. Tang, \u201cDeep learning face representation by joint identificationverification,\u201d in Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2014, pp. 1988\u20131996. [Online]. Available: http:\/\/papers.nips.cc\/paper\/5416-deep-learning-face-representation-by-joint-identification-verification.pdf"},{"key":"2022062314372015302_j_popets-2022-0008_ref_010","doi-asserted-by":"crossref","unstructured":"[10] Y. Sun, X. Wang, and X. Tang, \u201cDeep Learning Face Representation from Predicting 10,000 Classes,\u201d in 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, Jun. 2014, pp. 1891\u20131898. [Online]. Available: http:\/\/ieeexplore.ieee.org\/lpdocs\/epic03\/wrapper.htm?arnumber=690964010.1109\/CVPR.2014.244","DOI":"10.1109\/CVPR.2014.244"},{"key":"2022062314372015302_j_popets-2022-0008_ref_011","unstructured":"[11] \u2014\u2014, \u201cDeeply learned face representations are sparse, selective, and robust,\u201d in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, Jun. 2015, pp. 2892\u20132900. [Online]. Available: http:\/\/ieeexplore.ieee.org\/document\/7298907\/"},{"key":"2022062314372015302_j_popets-2022-0008_ref_012","unstructured":"[12] Y. Sun, D. Liang, X. Wang, and X. Tang, \u201cDeepid3: Face recognition with very deep neural networks,\u201d arXiv preprint arXiv:1502.00873, vol. abs\/1502.00873, 2015."},{"key":"2022062314372015302_j_popets-2022-0008_ref_013","doi-asserted-by":"crossref","unstructured":"[13] L. Huang, A. D. Joseph, B. Nelson, B. I. Rubinstein, and J. D. Tygar, \u201cAdversarial machine learning,\u201d in Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, ser. AISec \u201911. New York, NY, USA: Association for Computing Machinery, 2011, p. 43\u201358. [Online]. Available: https:\/\/doi.org\/10.1145\/2046684.204669210.1145\/2046684.2046692","DOI":"10.1145\/2046684.2046692"},{"key":"2022062314372015302_j_popets-2022-0008_ref_014","unstructured":"[14] I. J. Goodfellow, J. Shlens, and C. Szegedy, \u201cExplaining and harnessing adversarial examples,\u201d 2014."},{"key":"2022062314372015302_j_popets-2022-0008_ref_015","doi-asserted-by":"crossref","unstructured":"[15] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, \u201cDeepFool: A Simple and Accurate Method to Fool Deep Neural Networks,\u201d in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, Jun. 2016, pp. 2574\u20132582. [Online]. Available: http:\/\/ieeexplore.ieee.org\/document\/7780651\/10.1109\/CVPR.2016.282","DOI":"10.1109\/CVPR.2016.282"},{"key":"2022062314372015302_j_popets-2022-0008_ref_016","unstructured":"[16] A. Kurakin, I. Goodfellow, and S. Bengio, \u201cAdversarial machine learning at scale,\u201d 2016."},{"key":"2022062314372015302_j_popets-2022-0008_ref_017","unstructured":"[17] \u2014\u2014, \u201cAdversarial examples in the physical world,\u201d ICLR Workshop, 2017. [Online]. Available: https:\/\/arxiv.org\/abs\/1607.02533"},{"key":"2022062314372015302_j_popets-2022-0008_ref_018","unstructured":"[18] R. Awasthi, \u201cBreaking Deep Learning with Adversarial examples using Tensorflow,\u201d May 2018. [Online]. Available: https:\/\/cv-tricks.com\/how-to\/breaking-deep-learning-with-adversarial-examples-using-tensorflow\/"},{"key":"2022062314372015302_j_popets-2022-0008_ref_019","unstructured":"[19] Y. Liu, X. Chen, C. Liu, and D. Song, \u201cDelving into transferable adversarial examples and black-box attacks,\u201d CoRR, vol. abs\/1611.02770, 2016. [Online]. Available: http:\/\/arxiv.org\/abs\/1611.02770"},{"key":"2022062314372015302_j_popets-2022-0008_ref_020","unstructured":"[20] I. Evtimov, P. Sturmfels, and T. Kohno, \u201cFoggysight: A scheme for facial lookup privacy,\u201d CoRR, vol. abs\/2012.08588, 2020. [Online]. Available: https:\/\/arxiv.org\/abs\/2012.08588"},{"key":"2022062314372015302_j_popets-2022-0008_ref_021","unstructured":"[21] V. Cherepanova, M. Goldblum, H. Foley, S. Duan, J. P. Dickerson, G. Taylor, and T. Goldstein, \u201cLowkey: Leveraging adversarial attacks to protect social media users from facial recognition,\u201d in International Conference on Learning Representations, 2021. [Online]. Available: https:\/\/openreview.net\/forum?id=hJmtwocEqzc"},{"key":"2022062314372015302_j_popets-2022-0008_ref_022","unstructured":"[22] E. Radiya-Dixit and F. Tram\u00e8r, \u201cData poisoning won\u2019t save you from facial recognition,\u201d 2021."},{"key":"2022062314372015302_j_popets-2022-0008_ref_023","doi-asserted-by":"crossref","unstructured":"[23] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, \u201cImage Quality Assessment: From Error Visibility to Structural Similarity,\u201d IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600\u2013612, Apr. 2004. [Online]. Available: http:\/\/ieeexplore.ieee.org\/document\/1284395\/10.1109\/TIP.2003.819861","DOI":"10.1109\/TIP.2003.819861"},{"key":"2022062314372015302_j_popets-2022-0008_ref_024","unstructured":"[24] H.-W. Ng and S. Winkler, \u201cA data-driven approach to cleaning large face datasets,\u201d in 2014 IEEE International Conference on Image Processing (ICIP). Paris, France: IEEE, Oct. 2014, pp. 343\u2013347. [Online]. Available: http:\/\/ieeexplore.ieee.org\/document\/7025068\/"},{"key":"2022062314372015302_j_popets-2022-0008_ref_025","doi-asserted-by":"crossref","unstructured":"[25] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, \u201cJoint face detection and alignment using multitask cascaded convolutional networks,\u201d IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499\u20131503, 2016.","DOI":"10.1109\/LSP.2016.2603342"},{"key":"2022062314372015302_j_popets-2022-0008_ref_026","unstructured":"[26] B. Amos, B. Ludwiczuk, and M. Satyanarayanan, \u201cOpenface: A general-purpose face recognition library with mobile applications,\u201d CMU-CS-16-118, CMU School of Computer Science, Tech. Rep., 2016."},{"key":"2022062314372015302_j_popets-2022-0008_ref_027","unstructured":"[27] J. Liu, Y. Deng, T. Bai, and C. Huang, \u201cTargeting ultimate accuracy: Face recognition via deep embedding,\u201d CoRR, vol. abs\/1506.07310, 2015. [Online]. Available: http:\/\/arxiv.org\/abs\/1506.07310"},{"key":"2022062314372015302_j_popets-2022-0008_ref_028","doi-asserted-by":"crossref","unstructured":"[28] C. Ding and D. Tao, \u201cRobust face recognition via multimodal deep face representation,\u201d IEEE Transactions on Multimedia, vol. 17, no. 11, pp. 2049\u20132058, 2015.","DOI":"10.1109\/TMM.2015.2477042"},{"key":"2022062314372015302_j_popets-2022-0008_ref_029","unstructured":"[29] S. Sankaranarayanan, A. Alavi, and R. Chellappa, \u201cTriplet similarity embedding for face verification,\u201d CoRR, vol. abs\/1602.03418, 2016. [Online]. Available: http:\/\/arxiv.org\/abs\/1602.03418"},{"key":"2022062314372015302_j_popets-2022-0008_ref_030","doi-asserted-by":"crossref","unstructured":"[30] S. Sankaranarayanan, A. Alavi, C. D. Castillo, and R. Chellappa, \u201cTriplet probabilistic embedding for face verification and clustering,\u201d in 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS), 2016, pp. 1\u20138.10.1109\/BTAS.2016.7791205","DOI":"10.1109\/BTAS.2016.7791205"},{"key":"2022062314372015302_j_popets-2022-0008_ref_031","doi-asserted-by":"crossref","unstructured":"[31] X. Zhao, X. Liang, C. Zhao, M. Tang, and J. Wang, \u201cReal-Time Multi-Scale Face Detector on Embedded Devices,\u201d Sensors (Basel, Switzerland), vol. 19, no. 9, May 2019. [Online]. Available: https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC6539187\/10.3390\/s19092158653918731075955","DOI":"10.3390\/s19092158"},{"key":"2022062314372015302_j_popets-2022-0008_ref_032","unstructured":"[32] L. v. d. Maaten and G. Hinton, \u201cVisualizing Data using t-SNE,\u201d Journal of Machine Learning Research, vol. 9, no. Nov, pp. 2579\u20132605, 2008. [Online]. Available: https:\/\/www.jmlr.org\/papers\/v9\/vandermaaten08a.html"},{"key":"2022062314372015302_j_popets-2022-0008_ref_033","unstructured":"[33] G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, \u201cLabeled faces in the wild: A database for studying face recognition in unconstrained environments,\u201d University of Massachusetts, Amherst, Tech. Rep. 07-49, October 2007."},{"key":"2022062314372015302_j_popets-2022-0008_ref_034","doi-asserted-by":"crossref","unstructured":"[34] N. Pinto, Z. Stone, T. Zickler, and D. Cox, \u201cScaling up biologically-inspired computer vision: A case study in unconstrained face recognition on facebook,\u201d in CVPR 2011 WORKSHOPS, Jun. 2011, pp. 35\u201342, iSSN: 2160-7516.","DOI":"10.1109\/CVPRW.2011.5981788"},{"key":"2022062314372015302_j_popets-2022-0008_ref_035","unstructured":"[35] Systems Incorporated, Adobe, \u201cAdobe photoshop lightroom classic cc help,\u201d Tech. Rep., Feb. 2018. [Online]. Available: https:\/\/helpx.adobe.com\/pdf\/lightroom_reference.pdf"},{"key":"2022062314372015302_j_popets-2022-0008_ref_036","unstructured":"[36] \u201cFace Searching - Face++ Cognitive Services.\u201d [Online]. Available: https:\/\/www.faceplusplus.com\/face-searching\/"},{"key":"2022062314372015302_j_popets-2022-0008_ref_037","unstructured":"[37] N. Papernot, P. D. McDaniel, and I. J. Goodfellow, \u201cTransferability in machine learning: from phenomena to black-box attacks using adversarial samples,\u201d CoRR, vol. abs\/1605.07277, 2016. [Online]. Available: http:\/\/arxiv.org\/abs\/1605.07277"},{"key":"2022062314372015302_j_popets-2022-0008_ref_038","unstructured":"[38] D. Yi, Z. Lei, S. Liao, and S. Z. Li, \u201cLearning face representation from scratch,\u201d CoRR, vol. abs\/1411.7923, 2014. [Online]. Available: http:\/\/arxiv.org\/abs\/1411.7923"},{"key":"2022062314372015302_j_popets-2022-0008_ref_039","doi-asserted-by":"crossref","unstructured":"[39] I. William, D. R. Ignatius Moses Setiadi, E. H. Rachmawanto, H. A. Santoso, and C. A. Sari, \u201cFace recognition using facenet (survey, performance test, and comparison),\u201d in 2019 Fourth International Conference on Informatics and Computing (ICIC), 2019, pp. 1\u20136.10.1109\/ICIC47613.2019.8985786","DOI":"10.1109\/ICIC47613.2019.8985786"},{"key":"2022062314372015302_j_popets-2022-0008_ref_040","unstructured":"[40] Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman, \u201cVggface2: A dataset for recognising faces across pose and age,\u201d CoRR, vol. abs\/1710.08092, 2017. [Online]. Available: http:\/\/arxiv.org\/abs\/1710.08092"},{"key":"2022062314372015302_j_popets-2022-0008_ref_041","doi-asserted-by":"crossref","unstructured":"[41] O. M. Parkhi, A. Vedaldi, and A. Zisserman, \u201cDeep face recognition,\u201d in Proceedings of the British Machine Vision Conference (BMVC), X. Xie, M. W. Jones, and G. K. L. Tam, Eds. BMVA Press, September 2015, pp. 41.1\u201341.12. [Online]. Available: https:\/\/dx.doi.org\/10.5244\/C.29.4110.5244\/C.29.41","DOI":"10.5244\/C.29.41"},{"key":"2022062314372015302_j_popets-2022-0008_ref_042","unstructured":"[42] A. Athalye, N. Carlini, and D. Wagner, \u201cObfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples,\u201d in Proceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, J. Dy and A. Krause, Eds., vol. 80. Stockholmsm\u00e4ssan, Stockholm Sweden: PMLR, 10\u201315 Jul 2018, pp. 274\u2013283. [Online]. Available: http:\/\/proceedings.mlr.press\/v80\/athalye18a.html"},{"key":"2022062314372015302_j_popets-2022-0008_ref_043","doi-asserted-by":"crossref","unstructured":"[43] U. Shaham, Y. Yamada, and S. Negahban, \u201cUnderstanding adversarial training: Increasing local stability of supervised models through robust optimization,\u201d Neurocomputing, vol. 307, pp. 195\u2013204, 2018. [Online]. Available: https:\/\/www.sciencedirect.com\/science\/article\/pii\/S092523121830455710.1016\/j.neucom.2018.04.027","DOI":"10.1016\/j.neucom.2018.04.027"},{"key":"2022062314372015302_j_popets-2022-0008_ref_044","unstructured":"[44] A. Paudice, L. Mu\u00f1oz-Gonz\u00e1lez, A. Gyorgy, and E. C. Lupu, \u201cDetection of adversarial training examples in poisoning attacks through anomaly detection,\u201d 2018."},{"key":"2022062314372015302_j_popets-2022-0008_ref_045","doi-asserted-by":"crossref","unstructured":"[45] B. Wang, Y. Yao, S. Shan, H. Li, B. Viswanath, H. Zheng, and B. Y. Zhao, \u201cNeural cleanse: Identifying and mitigating back-door attacks in neural networks,\u201d in 2019 IEEE Symposium on Security and Privacy (SP), 2019, pp. 707\u2013723.10.1109\/SP.2019.00031","DOI":"10.1109\/SP.2019.00031"},{"key":"2022062314372015302_j_popets-2022-0008_ref_046","unstructured":"[46] E. Zhou, Z. Cao, and Q. Yin, \u201cNaive-deep face recognition: Touching the limit of LFW benchmark or not?\u201d CoRR, vol. abs\/1501.04690, 2015. [Online]. Available: http:\/\/arxiv.org\/abs\/1501.04690"},{"key":"2022062314372015302_j_popets-2022-0008_ref_047","doi-asserted-by":"crossref","unstructured":"[47] N. Crosswhite, J. Byrne, C. Stauffer, O. Parkhi, Q. Cao, and A. Zisserman, \u201cTemplate adaptation for face verification and identification,\u201d Image and Vision Computing, vol. 79, pp. 35\u201348, 2018. [Online]. Available: https:\/\/www.sciencedirect.com\/science\/article\/pii\/S026288561830147110.1016\/j.imavis.2018.09.002","DOI":"10.1016\/j.imavis.2018.09.002"}],"container-title":["Proceedings on Privacy Enhancing Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.sciendo.com\/pdf\/10.2478\/popets-2022-0008","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,20]],"date-time":"2022-07-20T16:31:51Z","timestamp":1658334711000},"score":1,"resource":{"primary":{"URL":"https:\/\/petsymposium.org\/popets\/2022\/popets-2022-0008.php"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,20]]},"references-count":47,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,11,20]]},"published-print":{"date-parts":[[2022,1,1]]}},"alternative-id":["10.2478\/popets-2022-0008"],"URL":"https:\/\/doi.org\/10.2478\/popets-2022-0008","relation":{},"ISSN":["2299-0984"],"issn-type":[{"value":"2299-0984","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,20]]}}}