{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,25]],"date-time":"2024-08-25T07:48:46Z","timestamp":1724572126211},"reference-count":48,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2019,8,15]],"date-time":"2019-08-15T00:00:00Z","timestamp":1565827200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["17K12653; 15H01678; 15H03636; 16K00044; 19H04071"],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"We propose a new clustering method based on a deep neural network. Given an unlabeled dataset and the number of clusters, our method directly groups the dataset into the given number of clusters in the original space. We use a conditional discrete probability distribution defined by a deep neural network as a statistical model. Our strategy is first to estimate the cluster labels of unlabeled data points selected from a high-density region, and then to conduct semi-supervised learning to train the model by using the estimated cluster labels and the remaining unlabeled data points. Lastly, by using the trained model, we obtain the estimated cluster labels of all given unlabeled data points. The advantage of our method is that it does not require key conditions. Existing clustering methods with deep neural networks assume that the cluster balance of a given dataset is uniform. Moreover, it also can be applied to various data domains as long as the data is expressed by a feature vector. In addition, it is observed that our method is robust against outliers. Therefore, the proposed method is expected to perform, on average, better than previous methods. We conducted numerical experiments on five commonly used datasets to confirm the effectiveness of the proposed method.<\/jats:p>","DOI":"10.3390\/e21080795","type":"journal-article","created":{"date-parts":[[2019,8,15]],"date-time":"2019-08-15T15:11:00Z","timestamp":1565881860000},"page":"795","source":"Crossref","is-referenced-by-count":5,"title":["Spectral Embedded Deep Clustering"],"prefix":"10.3390","volume":"21","author":[{"given":"Yuichiro","family":"Wada","sequence":"first","affiliation":[{"name":"Graduate School of Information Science, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8601, Japan"}]},{"given":"Shugo","family":"Miyamoto","sequence":"additional","affiliation":[{"name":"Department of Systems Innovation, School of Engineering, The University of Tokyo, Hongo Campus, Eng. Bldg. No. 3, 2F, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan"}]},{"given":"Takumi","family":"Nakagama","sequence":"additional","affiliation":[{"name":"Department of Mathematical and Computing Science, School of Computing, Tokyo Institute of Technology, 2-12-1 Ookayama, Meguro-ku, Tokyo 152-8552, Japan"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-8704-4748","authenticated-orcid":false,"given":"L\u00e9o","family":"And\u00e9ol","sequence":"additional","affiliation":[{"name":"Computer Science Department, Sorbonne Universit\u00e9, 4 place Jussieu, 75005 Paris, France"},{"name":"RIKEN AIP, Nihonbashi 1-chome Mitsui Building, 15th floor, 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan"}]},{"given":"Wataru","family":"Kumagai","sequence":"additional","affiliation":[{"name":"RIKEN AIP, Nihonbashi 1-chome Mitsui Building, 15th floor, 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-6878-5850","authenticated-orcid":false,"given":"Takafumi","family":"Kanamori","sequence":"additional","affiliation":[{"name":"Department of Mathematical and Computing Science, School of Computing, Tokyo Institute of Technology, 2-12-1 Ookayama, Meguro-ku, Tokyo 152-8552, Japan"},{"name":"RIKEN AIP, Nihonbashi 1-chome Mitsui Building, 15th floor, 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2019,8,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Berkhin, P. (2006). A survey of clustering data mining techniques. Grouping Multidimensional Data, Springer.","DOI":"10.1007\/3-540-28349-8_2"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1109\/TNN.2005.845141","article-title":"Survey of clustering algorithms","volume":"16","author":"Xu","year":"2005","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_3","unstructured":"MacQueen, J. (July, January 21). Some methods for classification and analysis of multivariate observations. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Oakland, CA, USA."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1093\/biomet\/56.3.463","article-title":"Estimating the components of a mixture of normal distributions","volume":"56","author":"Day","year":"1969","journal-title":"Biometrika"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"780","DOI":"10.1109\/TNN.2002.1000150","article-title":"Mercer kernel-based clustering in feature space","volume":"13","author":"Girolami","year":"2002","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, J., Lee, J., and Zhang, C. (2003, January 17\u201319). Kernel trick embedded Gaussian mixture model. Proceedings of the International Conference on Algorithmic Learning Theory, Sapporo, Japan.","DOI":"10.1007\/978-3-540-39624-6_14"},{"key":"ref_7","unstructured":"Ng, A.Y., Jordan, M.I., and Weiss, Y. (2002). On spectral clustering: Analysis and an algorithm. Advances in Neural Information Processing Systems, Proceedings of the Neural Information Processing Systems, NIPS 2002, Vancouver, BC, Canada, 9\u201314 December 2002, MIT Press."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1016\/j.patrec.2009.09.011","article-title":"Data clustering: 50 years beyond K-means","volume":"31","author":"Jain","year":"2010","journal-title":"Pattern Recognit. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the dimensionality of data with neural networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"ref_11","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, Proceedings of the NIPS 2012: Neural Information Processing Systems Conference, Lake Tahoe, NV, USA, 3\u20136 December 2012, MIT Press."},{"key":"ref_12","unstructured":"Xie, J., Girshick, R., and Farhadi, A. (2016, January 19\u201324). Unsupervised deep embedding for clustering analysis. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yang, J., Parikh, D., and Batra, D. (2016, January 27\u201330). Joint unsupervised learning of deep representations and image clusters. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.556"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Jiang, Z., Zheng, Y., Tan, H., Tang, B., and Zhou, H. (2016). Variational deep embedding: An unsupervised and generative approach to clustering. arXiv.","DOI":"10.24963\/ijcai.2017\/273"},{"key":"ref_15","unstructured":"Hu, W., Miyato, T., Tokui, S., Matsumoto, E., and Sugiyama, M. (2017, January 6\u201311). Learning Discrete Representations via Information Maximizing Self-Augmented Training. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_16","unstructured":"Springenberg, J.T. (2015). Unsupervised and semi-supervised learning with categorical generative adversarial networks. arXiv."},{"key":"ref_17","unstructured":"Shaham, U., Stanton, K., Li, H., Nadler, B., Basri, R., and Kluger, Y. (2018). SpectralNet: Spectral Clustering using Deep Neural Networks. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chapelle, O., and Zien, A. (2005, January 6\u20138). Semi-Supervised Classification by Low Density Separation. Proceedings of the AISTATS, Bridgetown, Barbados.","DOI":"10.7551\/mitpress\/9780262033589.001.0001"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Tian, F., Gao, B., Cui, Q., Chen, E., and Liu, T.Y. (2014, January 27\u201331). Learning deep representations for graph clustering. Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, Qu\u00e9bec City, QC, Canada.","DOI":"10.1609\/aaai.v28i1.8916"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Huang, P., Huang, Y., Wang, W., and Wang, L. (2014, January 24\u201328). Deep embedding network for clustering. Proceedings of the 2014 22nd International Conference on Pattern Recognition, Stockholm, Sweden.","DOI":"10.1109\/ICPR.2014.272"},{"key":"ref_21","unstructured":"Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.patcog.2017.09.015","article-title":"Learning by coincidence: Siamese networks and common variable learning","volume":"74","author":"Shaham","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1007\/s11222-007-9033-z","article-title":"A tutorial on spectral clustering","volume":"17","year":"2007","journal-title":"Stat. Comput."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Scholkopf, B., and Smola, A.J. (2001). Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press.","DOI":"10.7551\/mitpress\/4175.001.0001"},{"key":"ref_25","unstructured":"Arthur, D., and Vassilvitskii, S. (2007, January 7\u20139). k-means++: The advantages of careful seeding. Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, New Orleans, LA."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1979","DOI":"10.1109\/TPAMI.2018.2858821","article-title":"Virtual adversarial training: A regularization method for supervised and semi-supervised learning","volume":"41","author":"Miyato","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_27","unstructured":"Bachman, P., Alsharif, O., and Precup, D. (2014). Learning with pseudo-ensembles. Advances in Neural Information Processing Systems, Proceedings of the NIPS 2014: Neural Information Processing Systems Conference, Montreal, QC, Canada, 8\u201313 December 2014, MIT Press."},{"key":"ref_28","unstructured":"Goodfellow, I.J., Shlens, J., and Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv."},{"key":"ref_29","unstructured":"Tao, Y., Takagi, K., and Nakata, K. (2018, January 14\u201316). RDEC: Integrating Regularization into Deep Embedded Clustering for Imbalanced Datasets. Proceedings of the Asian Conference on Machine Learning, Beijing, China."},{"key":"ref_30","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, John Wiley & Sons."},{"key":"ref_31","unstructured":"Dasgupta, S., and Kpotufe, S. (2014). Optimal rates for k-NN density and mode estimation. Advances in Neural Information Processing Systems, Proceedings of the NIPS 2014: Neural Information Processing Systems Conference, Montreal, QC, Canada, 8\u201313 December 2014, MIT Press."},{"key":"ref_32","unstructured":"Moscovich, A., Jaffe, A., and Nadler, B. (2016). Minimax-optimal semi-supervised regression on unknown manifolds. arXiv."},{"key":"ref_33","unstructured":"Har-Peled, S. (2016). Computing the k Nearest-Neighbors for all Vertices via Dijkstra. arXiv."},{"key":"ref_34","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_35","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_36","unstructured":"Grandvalet, Y., and Bengio, Y. (2005). Semi-supervised learning by entropy minimization. Advances in Neural Information Processing Systems, Proceedings of the 19th Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 5\u20138 December 2005, MIT Press."},{"key":"ref_37","unstructured":"Krause, A., Perona, P., and Gomes, R.G. (2010). Discriminative clustering by regularized information maximization. Advances in Neural Information Processing Systems, Proceedings of the 24th Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 6\u20139 December 2010, MIT Press."},{"key":"ref_38","unstructured":"Dong, W., Moses, C., and Li, K. (April, January 28). Efficient k-nearest neighbor graph construction for generic similarity measures. Proceedings of the 20th International Conference on World Wide Web, Hyderabad, India."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Wang, D., Shi, L., and Cao, J. (2013, January 7\u201310). Fast algorithm for approximate k-nearest neighbor graph construction. Proceedings of the 2013 IEEE 13th International Conference on Data Mining Workshops, Dallas, TX, USA.","DOI":"10.1109\/ICDMW.2013.50"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhang, Y.M., Huang, K., Geng, G., and Liu, C.L. (2013, January 23\u201327). Fast kNN graph construction with locality sensitive hashing. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Prague, Czech Republic.","DOI":"10.1007\/978-3-642-40991-2_42"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_42","first-page":"361","article-title":"Rcv1: A new benchmark collection for text categorization research","volume":"5","author":"Lewis","year":"2004","journal-title":"J. Mach. Learn. Res."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1002\/nav.3800020109","article-title":"The Hungarian method for the assignment problem","volume":"2","author":"Kuhn","year":"1955","journal-title":"Nav. Res. Logist. Q."},{"key":"ref_44","unstructured":"Hastie, T., Tibshirani, R., and Friedman, J. (2008). The Elements of Statistical Learning, Springer."},{"key":"ref_45","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel."},{"key":"ref_46","unstructured":"Ioffe, S., and Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7\u201313). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"39501","DOI":"10.1109\/ACCESS.2018.2855437","article-title":"A survey of clustering with deep learning: From the perspective of network architecture","volume":"6","author":"Min","year":"2018","journal-title":"IEEE Access"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/8\/795\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,22]],"date-time":"2024-07-22T02:01:33Z","timestamp":1721613693000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/8\/795"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,8,15]]},"references-count":48,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2019,8]]}},"alternative-id":["e21080795"],"URL":"https:\/\/doi.org\/10.3390\/e21080795","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,8,15]]}}}