{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:56:27Z","timestamp":1740149787689,"version":"3.37.3"},"reference-count":48,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,3,30]],"date-time":"2024-03-30T00:00:00Z","timestamp":1711756800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["12171454, U19B2940"],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","award":["1"],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["CA204120, HL161691"],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"In the classification task, label noise has a significant impact on models\u2019 performance, primarily manifested in the disruption of prediction consistency, thereby reducing the classification accuracy. This work introduces a novel prediction consistency regularization that mitigates the impact of label noise on neural networks by imposing constraints on the prediction consistency of similar samples. However, determining which samples should be similar is a primary challenge. We formalize the similar sample identification as a clustering problem and employ twin contrastive clustering (TCC) to address this issue. To ensure similarity between samples within each cluster, we enhance TCC by adjusting clustering prior to distribution using label information. Based on the adjusted TCC\u2019s clustering results, we first construct the prototype for each cluster and then formulate a prototype-based regularization term to enhance prediction consistency for the prototype within each cluster and counteract the adverse effects of label noise. We conducted comprehensive experiments using benchmark datasets to evaluate the effectiveness of our method under various scenarios with different noise rates. The results explicitly demonstrate the enhancement in classification accuracy. Subsequent analytical experiments confirm that the proposed regularization term effectively mitigates noise and that the adjusted TCC enhances the quality of similar sample recognition.<\/jats:p>","DOI":"10.3390\/e26040308","type":"journal-article","created":{"date-parts":[[2024,3,31]],"date-time":"2024-03-31T17:39:37Z","timestamp":1711906777000},"page":"308","source":"Crossref","is-referenced-by-count":1,"title":["Prediction Consistency Regularization for Learning with Noise Labels Based on Contrastive Clustering"],"prefix":"10.3390","volume":"26","author":[{"given":"Xinkai","family":"Sun","sequence":"first","affiliation":[{"name":"School of Mathematics Sciences, University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Key Laboratory of Big Data Mining and Knowledge Management, Chinese Academy of Sciences, Beijing 100049, China"}]},{"given":"Sanguo","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Mathematics Sciences, University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Key Laboratory of Big Data Mining and Knowledge Management, Chinese Academy of Sciences, Beijing 100049, China"}]},{"given":"Shuangge","family":"Ma","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, Yale School of Public Health, New Haven, CT 06510, USA"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2352","DOI":"10.1162\/neco_a_00990","article-title":"Deep convolutional neural networks for image classification: A comprehensive review","volume":"29","author":"Rawat","year":"2017","journal-title":"Neural Comput."},{"key":"ref_2","first-page":"71","article-title":"Machine learning text classification model with NLP approach","volume":"2","author":"Razno","year":"2019","journal-title":"Comput. Linguist. Intell. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1633","DOI":"10.1109\/JBHI.2017.2705583","article-title":"DeepPap: Deep convolutional networks for cervical cell classification","volume":"21","author":"Zhang","year":"2017","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_4","first-page":"845","article-title":"Classification in the presence of label noise: A survey","volume":"25","author":"Verleysen","year":"2013","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_5","unstructured":"Jiang, L., Huang, D., Liu, M., and Yang, W. (2020, January 13\u201318). Beyond synthetic noise: Deep learning on controlled noisy labels. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yi, G.Y. (2017). Statistical Analysis with Measurement Error or Misclassification: Strategy, Method and Application, Springer.","DOI":"10.1007\/978-1-4939-6640-0"},{"key":"ref_7","unstructured":"Englesson, E., and Azizpour, H. (2021, January 6\u201314). Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels. Proceedings of the Advances in Neural Information Processing Systems, Virtual."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Iscen, A., Valmadre, J., Arnab, A., and Schmid, C. (2022, January 18\u201324). Learning with neighbor consistency for noisy labels. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00463"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020, January 13\u201319). Momentum Contrast for Unsupervised Visual Representation Learning. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"ref_10","unstructured":"Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. (2020, January 6\u201312). Unsupervised Learning of Visual Features by Contrasting Cluster Assignments. Proceedings of the Advances in Neural Information Processing Systems, Virtual."},{"key":"ref_11","unstructured":"Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020, January 13\u201318). A Simple Framework for Contrastive Learning of Visual Representations. Proceedings of the 37th International Conference on Machine Learning, Virtual."},{"key":"ref_12","first-page":"27734","article-title":"You never cluster alone","volume":"34","author":"Shen","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ghosh, A., Kumar, H., and Sastry, P.S. (2017, January 4\u20139). Robust loss functions under label noise for deep neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10894"},{"key":"ref_14","unstructured":"Wang, X., Hua, Y., Kodirov, E., Clifton, D.A., and Robertson, N.M. (2023, January 4). IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude\u2019s Variance Matters. Proceedings of the ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models, Hybrid Mode (in-Person and Virtual Attendance). Available online: https:\/\/rtml-iclr2023.github.io\/."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.neucom.2022.01.031","article-title":"Multi-category classification with label noise by robust binary loss","volume":"482","author":"Liu","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_16","unstructured":"Zhang, Z., and Sabuncu, M. (2018, January 3\u20138). Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wang, Y., Ma, X., Chen, Z., Luo, Y., Yi, J., and Bailey, J. (November, January 27). Symmetric Cross Entropy for Robust Learning With Noisy Labels. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.","DOI":"10.1109\/ICCV.2019.00041"},{"key":"ref_18","unstructured":"Liu, S., Niles-Weed, J., Razavian, N., and Fernandez-Granda, C. (2020, January 6\u201312). Early-Learning Regularization Prevents Memorization of Noisy Labels. Proceedings of the Advances in Neural Information Processing Systems, Virtual."},{"key":"ref_19","unstructured":"Li, M., Soltanolkotabi, M., and Oymak, S. (2020, January 26\u201328). Gradient descent with early stopping is provably robust to label noise for overparameterized neural networks. Proceedings of the International Conference on Artificial Intelligence and Statistics, Online."},{"key":"ref_20","unstructured":"Malach, E., and Shalev-Shwartz, S. (2024, March 28). Decoupling \u201cWhen to Update\u201d from \u201cHow to Update\u201d. Available online: https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/hash\/58d4d1e7b1e97b258c9ed0b37e02d087-Abstract.html."},{"key":"ref_21","unstructured":"Han, B., Yao, Q., Yu, X., Niu, G., Xu, M., Hu, W., Tsang, I., and Sugiyama, M. (2024, March 28). Co-Teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels. Available online: https:\/\/proceedings.neurips.cc\/paper\/2018\/hash\/a19744e268754fb0148b017647355b7b-Abstract.html."},{"key":"ref_22","unstructured":"Yu, X., Han, B., Yao, J., Niu, G., Tsang, I., and Sugiyama, M. (2019, January 9\u201315). How does disagreement help generalization against label corruption?. Proceedings of the International Conference on Machine Learning. PMLR, Long Beach, CA, USA."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wei, H., Feng, L., Chen, X., and An, B. (2020, January 14\u201319). Combating noisy labels by agreement: A joint training method with co-regularization. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01374"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Sarfraz, F., Arani, E., and Zonooz, B. (2021, January 5\u20139). Noisy concurrent training for efficient learning under label noise. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Virtual.","DOI":"10.1109\/WACV48630.2021.00320"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Tan, C., Xia, J., Wu, L., and Li, S.Z. (2021, January 20\u201324). Co-learning: Learning from noisy labels with self-supervision. Proceedings of the 29th ACM International Conference on Multimedia, Virtual.","DOI":"10.1145\/3474085.3475622"},{"key":"ref_26","first-page":"21271","article-title":"Bootstrap your own latent-a new approach to self-supervised learning","volume":"33","author":"Grill","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, X., and He, K. (2021, January 20\u201325). Exploring simple siamese representation learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01549"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1007\/978-3-030-01264-9_9","article-title":"Deep Clustering for Unsupervised Learning of Visual Features","volume":"Volume 11218","author":"Caron","year":"2018","journal-title":"Computer Vision\u2014ECCV 2018"},{"key":"ref_29","unstructured":"Li, J., Zhou, P., Xiong, C., and Hoi, S. (2020, January 26\u201330). Prototypical Contrastive Learning of Unsupervised Representations. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zheltonozhskii, E., Baskin, C., Mendelson, A., Bronstein, A.M., and Litany, O. (2022, January 4\u20138). Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labels. Proceedings of the 2022 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV51458.2022.00046"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ghosh, A., and Lan, A. (2021, January 19\u201325). Contrastive Learning Improves Model Robustness Under Label Noise. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN, USA.","DOI":"10.1109\/CVPRW53098.2021.00304"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ortego, D., Arazo, E., Albert, P., O\u2019Connor, N.E., and McGuinness, K. (2021, January 20\u201325). Multi-Objective Interpolation Training for Robustness to Label Noise. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00654"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Li, S., Xia, X., Ge, S., and Liu, T. (2022, January 18\u201324). Selective-supervised contrastive learning with noisy labels. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00041"},{"key":"ref_34","unstructured":"Li, J., Xiong, C., and Hoi, S.C. (2021, January 3\u20137). MoPro: Webly Supervised Learning with Momentum Prototypes. Proceedings of the International Conference on Learning Representations, Virtual."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Li, J., Xiong, C., and Hoi, S.C. (2021, January 11\u201317). Learning from Noisy Data with Robust Representation Learning. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00935"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Huang, Z., Zhang, J., and Shan, H. (2023, January 17\u201324). Twin Contrastive Learning with Noisy Labels. Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01122"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Yi, L., Liu, S., She, Q., McLeod, A.I., and Wang, B. (2022, January 18\u201324). On learning contrastive representations for learning with noisy labels. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01618"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1109\/MSP.2021.3134634","article-title":"Self-supervised representation learning: Introduction, advances, and challenges","volume":"39","author":"Ericsson","year":"2022","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_39","unstructured":"Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv."},{"key":"ref_40","unstructured":"Jang, E., Gu, S., and Poole, B. (2016, January 2\u20134). Categorical Reparameterization with Gumbel-Softmax. Proceedings of the International Conference on Learning Representations, San Juan, Puerto Rico."},{"key":"ref_41","unstructured":"Krizhevsky, A. (2024, March 28). Learning Multiple Layers of Features from Tiny Images. Available online: http:\/\/www.cs.utoronto.ca\/~kriz\/learning-features-2009-TR.pdf."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Patrini, G., Rozza, A., Krishna Menon, A., Nock, R., and Qu, L. (2017, January 21\u201326). Making deep neural networks robust to label noise: A loss correction approach. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.240"},{"key":"ref_44","unstructured":"Song, H., Kim, M., and Lee, J.G. (2019, January 9\u201315). Selfie: Refurbishing unclean samples for robust deep learning. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_45","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Hinton","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"4380","DOI":"10.1093\/bioinformatics\/btac520","article-title":"CoGO: A contrastive learning framework to predict disease similarity based on gene network and ontology structure","volume":"38","author":"Chen","year":"2022","journal-title":"Bioinformatics"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1093\/bioinformatics\/btab718","article-title":"Accurate inference of gene regulatory interactions from spatial gene expression with deep contrastive learning","volume":"38","author":"Zheng","year":"2022","journal-title":"Bioinformatics"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Cai, D., Sun, C., Song, M., Zhang, B., Hong, S., and Li, H. (2022, January 28\u201330). Hypergraph contrastive learning for electronic health records. Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), Alexandria, VA, USA.","DOI":"10.1137\/1.9781611977172.15"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/4\/308\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,3]],"date-time":"2024-04-03T04:36:20Z","timestamp":1712118980000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/4\/308"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,30]]},"references-count":48,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["e26040308"],"URL":"https:\/\/doi.org\/10.3390\/e26040308","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2024,3,30]]}}}