Abstract
In this paper, we conducted the ImageNet Reannotation workshop with researchers who use ImageNet to find doubtful data in ImageNet. Recent great growth of deep learning is supported by large scale datasets collected by cloud working such as ImageNet, but it seems to have not so few doubtful data for given tasks. We assume that the professionals can efficiently and accurately find doubtful data while they know what kind of data would be better for learning classification tasks. Moreover, we adopted a group working scheme so that it could be more efficient and accurate. This paper shows the re-annotation result that clarifies category and reason of doubtfulness in the large scale dataset constructed by cloud workers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Beyer, L., Hénaff, O.J., Kolesnikov, A., Zhai, X., van den Oord, A.: Are we done with ImageNet? arXiv preprint (2020). arXiv:2006.07159
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848
Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., Song, D.: Natural adversarial examples. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021), pp. 15262–15271 (2021)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25. Curran Associates, Inc. (2012)
Northcutt, C.G., Athalye, A., Mueller, J.: Pervasive label errors in test sets destabilize machine learning benchmarks. In: Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks (2021)
Recht, B., Roelofs, R., Schmidt, L., Shankar, V.: Do ImageNet classifier generalize to ImageNet? In: Proceedings of the 36th International Conference on Machine Learning (ICML 2019) (2019)
Tsipras, D., Santurkar, S., Engstrom, L., Ilyas, A., Madry, A.: From imagenet to image classification: contextualizing progress on benchmarks. In: ArXiv preprint arXiv:2005.11295 (2020)
Yang, K., Qinami, K., Fei-Fei, L., Deng, J., Russakovsky, O.: Towards fairer datasets: filtering and balancing the distribution of the people subtree in the ImageNet hierarchy. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* 2020, pp. 547–558. Association for Computing Machinery, New York (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Suzuki, R., Kataoka, H. (2022). Revealing Doubtful Data in 200k Images via Re-annotation Workshop by Researcher Community. In: Stephanidis, C., Antona, M., Ntoa, S., Salvendy, G. (eds) HCI International 2022 – Late Breaking Posters. HCII 2022. Communications in Computer and Information Science, vol 1655. Springer, Cham. https://doi.org/10.1007/978-3-031-19682-9_88
Download citation
DOI: https://doi.org/10.1007/978-3-031-19682-9_88
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19681-2
Online ISBN: 978-3-031-19682-9
eBook Packages: Computer ScienceComputer Science (R0)