{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T13:11:38Z","timestamp":1740143498178,"version":"3.37.3"},"reference-count":66,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2019,9,23]],"date-time":"2019-09-23T00:00:00Z","timestamp":1569196800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Oculus"},{"DOI":"10.13039\/100004318","name":"Microsoft","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004318","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Appl. Percept."],"published-print":{"date-parts":[[2019,10,31]]},"abstract":"Scene recognition is an essential component of both machine and biological vision. Recent advances in computer vision using deep convolutional neural networks (CNNs) have demonstrated impressive sophistication in scene recognition, through training on large datasets of labeled scene images (Zhou et al. 2018, 2014). One criticism of CNN-based approaches is that performance may not generalize well beyond the training image set (Torralba and Efros 2011), and may be hampered by minor image modifications, which in some cases are barely perceptible to the human eye (Goodfellow et al. 2015; Szegedy et al. 2013). While these \u201cadversarial examples\u201d may be unlikely in natural contexts, during many real-world visual tasks scene information can be degraded or limited due to defocus blur, camera motion, sensor noise, or occluding objects. Here, we quantify the impact of several image degradations (some common, and some more exotic) on indoor\/outdoor scene classification using CNNs. For comparison, we use human observers as a benchmark, and also evaluate performance against classifiers using limited, manually selected descriptors. While the CNNs outperformed the other classifiers and rivaled human accuracy for intact images, our results show that their classification accuracy is more affected by image degradations than human observers. On a practical level, however, accuracy of the CNNs remained well above chance for a wide range of image manipulations that disrupted both local and global image statistics. We also examine the level of image-by-image agreement with human observers, and find that the CNNs\u2019 agreement with observers varied as a function of the nature of image manipulation. In many cases, this agreement was not substantially different from the level one would expect to observe for two independent classifiers. Together, these results suggest that CNN-based scene classification techniques are relatively robust to several image degradations. However, the pattern of classifications obtained for ambiguous images does not appear to closely reflect the strategies employed by human observers.<\/jats:p>","DOI":"10.1145\/3342349","type":"journal-article","created":{"date-parts":[[2019,9,23]],"date-time":"2019-09-23T12:10:12Z","timestamp":1569240612000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":14,"title":["Assessing Neural Network Scene Classification from Degraded Images"],"prefix":"10.1145","volume":"16","author":[{"given":"Timothy","family":"Tadros","sequence":"first","affiliation":[{"name":"University of California, San Diego, La Jolla, CA, USA"}]},{"given":"Nicholas C.","family":"Cullen","sequence":"additional","affiliation":[{"name":"University of Pennsylvania, Philadelphia, PA, USA"}]},{"given":"Michelle R.","family":"Greene","sequence":"additional","affiliation":[{"name":"Bates College, Lewiston, ME, USA"}]},{"given":"Emily A.","family":"Cooper","sequence":"additional","affiliation":[{"name":"University of California, Berkeley, Berkeley, CA, USA"}]}],"member":"320","published-online":{"date-parts":[[2019,9,23]]},"reference":[{"key":"e_1_2_2_1_1","first-page":"15","article-title":"Pixels to voxels: Modeling visual representation in the human brain. Retrieved from","volume":"1407","author":"Agrawal Pulkit","year":"2014","journal-title":"Arxiv Preprint Arxiv"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1995.tb02031.x"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.177.4043.77"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0033776"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.22"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.visres.2013.04.006"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1003963"},{"key":"e_1_2_2_8_1","first-page":"15","article-title":"Deep neural networks predict hierarchical spatio-temporal cortical dynamics of human visual object recognition. Retrieved from","volume":"1601","author":"Cichy Radoslaw M.","year":"2016","journal-title":"Arxiv Preprint Arxiv"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/QoMEX.2016.7498955"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306241"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1207\/S15327590IJHC1502_3"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(88)90014-7"},{"volume-title":"Wichmann","year":"2018","author":"Geirhos Robert","key":"e_1_2_2_13_1"},{"key":"e_1_2_2_14_1","first-page":"1","article-title":"Explaining and harnessing adversarial examples. Retrieved from","volume":"1412","author":"Goodfellow Ian J.","year":"2015","journal-title":"Arxiv Preprint Arxiv"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2013.00777"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1037\/xge0000129"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1006327"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9280.2009.02316.x"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cogpsych.2008.06.001"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.32962"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2016.0102"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.5023-14.2015"},{"volume-title":"Proceedings of the 34th International Conference on Machine Learning-Volume 70","author":"Guo Chuan","key":"e_1_2_2_23_1"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654889"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2017.01726"},{"volume-title":"Proceedings of the International Conference of the Biometrics Special Interest Group (BIOSIG\u201916)","author":"Karahan Samil","key":"e_1_2_2_26_1"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.2307\/2530610"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1003915"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.2307\/2531405"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-vision-082114-035447"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.3389\/neuro.01.016.2008"},{"volume-title":"Proceedings of the Conference on Advances in Neural Information Processing Systems 25","author":"Krizhevsky Alex","key":"e_1_2_2_32_1"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1004896"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.68"},{"volume-title":"Deep learning. Nature 521, 7553","year":"2015","author":"LeCun Yann","key":"e_1_2_2_35_1"},{"volume-title":"Proceedings of the Conference on Advances in Neural Information Processing Systems 23","author":"Li Li-Jia","key":"e_1_2_2_36_1"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063176.2063200"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1878803.1878851"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2001.937655"},{"key":"e_1_2_2_40_1","unstructured":"Microsoft. [n. d.]. Seeing AI. Retrieved from: https:\/\/www.microsoft.com\/en-us\/seeing-ai. Microsoft. [n. d.]. Seeing AI. Retrieved from: https:\/\/www.microsoft.com\/en-us\/seeing-ai."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1011139631724"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126283"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1167\/16.7.12"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.0388-18.2018"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.visres.2004.04.006"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03207704"},{"key":"e_1_2_2_48_1","first-page":"1","article-title":"Intriguing properties of neural networks. Retrieved from","volume":"1312","author":"Szegedy Christian","year":"2013","journal-title":"Arxiv Preprint Arxiv"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1167\/7.14.4"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0952523808080930"},{"volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201911)","author":"Torralba Antonio","key":"e_1_2_2_51_1"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1088\/0954-898X_14_3_302"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0285(83)90006-3"},{"key":"e_1_2_2_54_1","first-page":"10","article-title":"Examining the impact of blur on recognition by convolutional networks. Retrieved from","volume":"1611","author":"Vasiljevic Igor","year":"2016","journal-title":"Arxiv Preprint Arxiv"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/1278387.1278393"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1177\/0956797613512662"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvcir.2013.11.005"},{"key":"e_1_2_2_58_1","first-page":"3596","article-title":"A data driven approach to understanding the organization of high-level visual cortex. Sci","volume":"7","author":"Watson David M.","year":"2017","journal-title":"Rep."},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5539970"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1038\/nn.4244"},{"volume-title":"Proceedings of the Conference on Advances in Neural Information Processing Systems 26","author":"Yamins Daniel L.","key":"e_1_2_2_61_1"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2971648.2971730"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.319"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2723009"},{"volume-title":"Proceedings of the Conference on Advances in Neural Information Processing Systems 27","year":"2014","author":"Zhou Bolei","key":"e_1_2_2_65_1"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2017.7952349"}],"container-title":["ACM Transactions on Applied Perception"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3342349","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T09:20:53Z","timestamp":1672564853000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3342349"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,23]]},"references-count":66,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,10,31]]}},"alternative-id":["10.1145\/3342349"],"URL":"https:\/\/doi.org\/10.1145\/3342349","relation":{},"ISSN":["1544-3558","1544-3965"],"issn-type":[{"type":"print","value":"1544-3558"},{"type":"electronic","value":"1544-3965"}],"subject":[],"published":{"date-parts":[[2019,9,23]]},"assertion":[{"value":"2018-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-09-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}