Learning a Dynamic Privacy-Preserving Camera Robust to Inversion Attacks | SpringerLink
Skip to main content
Springer Nature Link
Log in

Learning a Dynamic Privacy-Preserving Camera Robust to Inversion Attacks

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15128))

Included in the following conference series:

  • 288 Accesses

Abstract

The problem of designing a privacy-preserving camera (PPC) is considered. Previous designs rely on a static point spread function (PSF), optimized to prevent detection of private visual information, such as recognizable facial features. However, the PSF can be easily recovered by measuring the camera response to a point light source, making these cameras vulnerable to PSF inversion attacks. A new dynamic privacy-preserving (DyPP) camera design is proposed to prevent such attacks. DyPP cameras rely on dynamic optical elements, such spatial light modulators, to implement a time-varying PSF, which changes from picture to picture. PSFs are drawn randomly with a learned manifold embedding, trained adversarially to simultaneously meet user-specified targets for privacy, such as face recognition accuracy, and task utility. Empirical evaluations on multiple privacy-preserving vision tasks demonstrate that the DyPP design is significantly more robust to PSF inversion attacks than previous PPCs. Furthermore, the hardware feasibility of the approach is validated by a proof-of-concept camera model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 8465
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 10581
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    This assumes that standard protections are used to prevent attackers from hacking into the camera or recovering camera parameters if they do.

  2. 2.

    While we focus on face recognition to measure privacy risk, the proposed camera design can be trivially extended to other privacy criteria, such as age, gender, or race classification accuracy.

  3. 3.

    We ignored 25 identities without enough face image samples, due to invalid URLs.

References

  1. Baek, S.H., et al.: Single-shot hyperspectral-depth imaging with learned diffractive optics. In: ICCV (2021)

    Google Scholar 

  2. Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., Nayar, S.K.: Face Swapping: automatically replacing faces in photographs. ACM Trans. Graph. (ToG) (2008)

    Google Scholar 

  3. Bojarski, M., et al.: End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016)

  4. Bouchabou, D., Nguyen, S.M., Lohr, C., LeDuc, B., Kanellos, I.: A survey of human activity recognition in smart homes based on iot sensors algorithms: taxonomies, challenges, and opportunities with deep learning. Sensors 21(18), 6037 (2021)

    Article  Google Scholar 

  5. Chakrabarti, A.: Learning sensor multiplexing design through back-propagation. In: NeurIPS (2016)

    Google Scholar 

  6. Chan, A.B., Liang, Z.S.J., Vasconcelos, N.: Privacy preserving crowd monitoring: counting people without people models or tracking. In: CVPR, pp. 1–7. IEEE (2008)

    Google Scholar 

  7. Chang, J., Sitzmann, V., Dun, X., Heidrich, W., Wetzstein, G.: Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification. Sci. Rep. 8(1), 12324 (2018)

    Article  Google Scholar 

  8. Chang, J., Wetzstein, G.: Deep optics for monocular depth estimation and 3d object detection. In: ICCV (2019)

    Google Scholar 

  9. Chinomi, K., Nitta, N., Ito, Y., Babaguchi, N.: PriSurv: privacy protected video surveillance system using adaptive visual abstraction. In: Satoh, S., Nack, F., Etoh, M. (eds.) MMM 2008. LNCS, vol. 4903, pp. 144–154. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-77409-9_14

    Chapter  Google Scholar 

  10. Chugunov, I., Baek, S.H., Fu, Q., Heidrich, W., Heide, F.: Mask-tof: learning microlens masks for flying pixel correction in time-of-flight imaging. In: CVPR (2021)

    Google Scholar 

  11. Cover, T.M., Thomas, J.A.: Elements of Information Theory. John Wiley & Sons, Hoboken (1999)

    Google Scholar 

  12. Criminisi, A., Perez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: CVPR (2003)

    Google Scholar 

  13. Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. (TIP) 13(9), 1200–1212 (2004)

    Article  Google Scholar 

  14. Deng, J., Guo, J., Xue, N., Zafeiriou, S.: Arcface: additive angular margin loss for deep face recognition. In: CVPR (2019)

    Google Scholar 

  15. Ding, X., Lin, Z., He, F., Wang, Y., Huang, Y.: A deeply-recursive convolutional network for crowd counting. In: ICASSP (2018)

    Google Scholar 

  16. Dong, J., Roth, S., Schiele, B.: Deep wiener deconvolution: wiener meets deep learning for image deblurring. In: NeurIPS (2020)

    Google Scholar 

  17. Duta, I.C., Liu, L., Zhu, F., Shao, L.: Improved residual networks for image and video recognition. In: ICPR. IEEE (2021)

    Google Scholar 

  18. Fan, J., Luo, H., Hacid, M.S., Bertino, E.: A novel approach for privacy-preserving video sharing. In: ACM International Conference on Information and Knowledge Management (CIKM) (2005)

    Google Scholar 

  19. Frome, A., et al.: Large-scale privacy protection in google street view. In: ICCV (2009)

    Google Scholar 

  20. Gao, H., Tao, X., Shen, X., Jia, J.: Dynamic scene deblurring with parameter selective sharing and nested skip connections. In: CVPR (2019)

    Google Scholar 

  21. Goodman, J.W.: Introduction to Fourier Optics, 4th edn. W. H. Freeman (2017)

    Google Scholar 

  22. Grigorescu, S., Trasnea, B., Cocias, T., Macesanu, G.: A survey of deep learning techniques for autonomous driving. J. Field Rob. 37(3), 362–386 (2020)

    Article  Google Scholar 

  23. Harwit, M., Sloane, N.J.A.: Hadamard transform optics (1979)

    Google Scholar 

  24. Rishabh, S., Mayank, P., Swapnil, G.: Smart home automation using computer vision and segmented image processing. In: 2019 International Conference on Communication and Signal Processing (ICCSP) (2019)

    Google Scholar 

  25. Hassan, R.H., Shaffer, P., Crandall, D., Apu Kapadia, E.T.: Cartooning for enhanced privacy in lifelogging and streaming videos. In: CVPR Workshops (2017)

    Google Scholar 

  26. He, L., Wang, G., Hu, Z.: Learning depth from single images with deep neural network embedding focal length. IEEE Trans. Image Process. (TIP) 27(9), 4676–4689 (2018)

    Article  MathSciNet  Google Scholar 

  27. Hershko, E., Weiss, L.E., Michaeli, T., Shechtman, Y.: Multicolor localization microscopy and point-spread-function engineering by deep learning. Opt. Express 27(5), 6158–6183 (2019)

    Article  Google Scholar 

  28. Hinojosa, C., Marquez, M., Arguello, H., Adeli, E., Fei-Fei, L., Niebles, J.C.: Privhar: Recognizing human actions from privacy-preserving lens. In: ECCV 2022. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-19772-7_19

  29. Hinojosa, C., Niebles, J.C., Arguello, H.: Learning privacy-preserving optics for human pose estimation. In: ICCV (2021)

    Google Scholar 

  30. Hu, Y., et al.: Planning-oriented autonomous driving. In: CVPR (2023)

    Google Scholar 

  31. Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst (2007)

    Google Scholar 

  32. Jeon, D.S., et al.: Compact snapshot hyperspectral imaging with diffracted rotation. ACM Trans. Graph. (ToG) 38(4) (2019)

    Google Scholar 

  33. Kellman, M., Bostan, E., Chen, M., Waller, L.: Data-driven design for fourier ptychographic microscopy. In: IEEE International Conference on Computational Photography (ICCP), pp. 1–8. IEEE (2019)

    Google Scholar 

  34. Kitahara, I., Kogure, K., Hagita, N.: Stealth vision for protecting privacy. In: ICPR, vol. 4, pp. 404–407. IEEE (2004)

    Google Scholar 

  35. Kumar, N., Berg, A.C., Belhumeur, P.N., Nayar, S.K.: Attribute and simile classifiers for face verification. In: ICCV (2009)

    Google Scholar 

  36. Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D., Matas, J.: Deblurgan: blind motion deblurring using conditional adversarial networks. In: CVPR (2018)

    Google Scholar 

  37. Li, L., Wang, L., Song, W., Zhang, L., Xiong, Z., Huang, H.: Quantization-aware deep optics for diffractive snapshot hyperspectral imaging. In: CVPR (2022)

    Google Scholar 

  38. Lin, H., Ma, Z., Ji, R., Wang, Y., Hong, X.: Boosting crowd counting via multifaceted attention. In: CVPR (2022)

    Google Scholar 

  39. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  40. Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: Sphereface: deep hypersphere embedding for face recognition. In: CVPR (2017)

    Google Scholar 

  41. Maji, D., Nagori, S., Mathew, M., Poddar, D.: Yolo-pose: enhancing yolo for multi person pose estimation using object keypoint similarity loss. In: CVPRW (2022)

    Google Scholar 

  42. Marco, J., et al.: Deeptof: off-the-shelf real-time correction of multipath interference in time-of-flight imaging. ACM Trans. Graph. (ToG) 36(6), 1–12 (2017)

    Article  Google Scholar 

  43. Metzler, C.A., Ikoma, H., Peng, Y., Wetzstein, G.: Deep optics for single-shot high-dynamic-range imaging. In: CVPR (2020)

    Google Scholar 

  44. Moschoglou, S., Papaioannou, A., Sagonas, C., Deng, J., Kotsia, I., Zafeiriou, S.: Agedb: the first manually collected, in-the-wild age database. In: CVPRW (2017)

    Google Scholar 

  45. Neustaedter, C., Greenberg, S., Boyle, M.: Blur filtration fails to preserve privacy for home-based video conferencing. ACM Trans. Comput.-Hum. Interact. 13(1), 1–36 (2006)

    Google Scholar 

  46. Nimisha, T.M., Kumar Singh, A., Rajagopalan, A.N.: Blur-invariant deep learning for blind-deblurring. In: ICCV (2017)

    Google Scholar 

  47. Noll, R.J.: Zernike polynomials and atmospheric turbulence. J. Opt. Soc. Am. 66(3), 207–211 (1976)

    Article  MathSciNet  Google Scholar 

  48. Orekondy, T., Schiele, B., Fritz, M.: Towards a visual privacy advisor: understanding and predicting privacy risks in images. In: ICCV (2017)

    Google Scholar 

  49. Padilla-López, J.R., Chaaraoui, A.A., Flórez-Revuelta, F.: Visual privacy protection methods: a survey. Expert Syst. Appl. 42(9), 4177–4195 (2015)

    Article  Google Scholar 

  50. Pichler, G., Colombo, P.J.A., Boudiaf, M., Koliander, G., Piantanida, P.: A differential entropy estimator for training neural networks. In: ICML (2022)

    Google Scholar 

  51. Pittaluga, F., Koppal, S.J.: Privacy preserving optics for miniature vision sensors. In: CVPR (2015)

    Google Scholar 

  52. Pittaluga, F., Koppal, S.J.: Pre-capture privacy for small vision sensors. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 39(11), 2215–2226 (2016)

    Article  Google Scholar 

  53. Pittaluga, F., Zivkovic, A., Koppal, S.J.: Sensor-level privacy for thermal cameras. In: 2016 IEEE International Conference on Computational Photography (ICCP), pp. 1–12. IEEE (2016)

    Google Scholar 

  54. Ren, Z., Lee, Y.J., Ryoo, M.S.: Learning to anonymize faces for privacy preserving action detection. In: ECCV (2018)

    Google Scholar 

  55. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  56. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

    Article  MathSciNet  Google Scholar 

  57. Ryoo, M., Kim, K., Yang, H.: Extreme low resolution activity recognition with multi-siamese embedding learning. In: AAAI (2018)

    Google Scholar 

  58. Ryoo, M., Rothrock, B., Fleming, C., Yang, H.J.: Privacy-preserving human activity recognition from extreme low resolution. In: AAAI (2017)

    Google Scholar 

  59. Sitzmann, V., et al.: End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging. ACM Trans. Graph. (TOG) 37(4), 1–13 (2018)

    Article  Google Scholar 

  60. Su, S., Heide, F., Wetzstein, G., Heidrich, W.: Deep end-to-end time-of-flight imaging. In: CVPR (2018)

    Google Scholar 

  61. Tasneem, Z., et al.: Learning phase mask for privacy-preserving passive depth estimation. In: ECCV 2022. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-20071-7_30

  62. Ultralytics: YOLOv5: A state-of-the-art real-time object detection system (2021). https://docs.ultralytics.com

  63. Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. In: CVPR (2018)

    Google Scholar 

  64. Vogel, C.R.: Computational methods for inverse problems. SIAM (2002)

    Google Scholar 

  65. Wetzstein, G., Ikoma, H., Metzler, C., Peng, Y.: Deep optics: learning cameras and optical computing systems. In: 2020 54th Asilomar Conference on Signals, Systems, and Computers, pp. 1313–1315. IEEE (2020)

    Google Scholar 

  66. Wu, Z., Wang, H., Wang, Z., Jin, H., Wang, Z.: Privacy-preserving deep action recognition: an adversarial learning framework and a new dataset. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 44(4), 2126–2139 (2020)

    Article  Google Scholar 

  67. Yu, J., de Antonio, A., Villalba-Mora, E.: Deep learning (CNN, RNN) applications for smart homes: a systematic review. Computers 11(2), 26 (2022)

    Article  Google Scholar 

  68. Zhang, Y., Zhou, D., Chen, S., Gao, S., Ma, Y.: Single-image crowd counting via multi-column convolutional neural network. In: CVPR (2016)

    Google Scholar 

Download references

Acknowledgements

We thank Carlos Hinojosa for sharing the PSF of PP-HPE. This work was partially funded by NSF award IIS-2303153, a gift from Qualcomm, and NVIDIA GPU donations. We also acknowledge and thank the use of the Nautilus platform for some of the experiments discussed above.

Author information

Authors and Affiliations

  1. University of California, San Diego, USA

    Jiacheng Cheng, Xiang Dai, Nick Antipa & Nuno Vasconcelos

  2. Harbin Institute of Technology, Shenzhen, China

    Jia Wan

Authors
  1. Jiacheng Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jia Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nick Antipa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nuno Vasconcelos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiacheng Cheng .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Aleš Leonardis

  2. University of Trento, Trento, Italy

    Elisa Ricci

  3. Technical University of Darmstadt, Darmstadt, Germany

    Stefan Roth

  4. Princeton University, Princeton, NJ, USA

    Olga Russakovsky

  5. Czech Technical University in Prague, Prague, Czech Republic

    Torsten Sattler

  6. École des Ponts ParisTech, Marne-la-Vallée, France

    Gül Varol

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 8334 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cheng, J., Dai, X., Wan, J., Antipa, N., Vasconcelos, N. (2025). Learning a Dynamic Privacy-Preserving Camera Robust to Inversion Attacks. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15128. Springer, Cham. https://doi.org/10.1007/978-3-031-72897-6_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72897-6_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72896-9

  • Online ISBN: 978-3-031-72897-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics