Data depth based clustering analysis

Jeong, Myeong -Hun; Cai, Yaping; Sullivan, Clair J.; Wang, Shaowen

doi:10.1145/2996913.2996984

Data depth based clustering analysis

Conference · Fri Jan 01 00:00:00 EST 2016

DOI:https://doi.org/10.1145/2996913.2996984· OSTI ID:1438413

Jeong, Myeong -Hun ^[1]; Cai, Yaping ^[1]; Sullivan, Clair J. ^[1]; Wang, Shaowen ^[1]

Univ. of Illinois at Urbana-Champaign, Urbana, IL (United States)

Here, this paper proposes a new algorithm for identifying patterns within data, based on data depth. Such a clustering analysis has an enormous potential to discover previously unknown insights from existing data sets. Many clustering algorithms already exist for this purpose. However, most algorithms are not affine invariant. Therefore, they must operate with different parameters after the data sets are rotated, scaled, or translated. Further, most clustering algorithms, based on Euclidean distance, can be sensitive to noises because they have no global perspective. Parameter selection also significantly affects the clustering results of each algorithm. Unlike many existing clustering algorithms, the proposed algorithm, called data depth based clustering analysis (DBCA), is able to detect coherent clusters after the data sets are affine transformed without changing a parameter. It is also robust to noises because using data depth can measure centrality and outlyingness of the underlying data. Further, it can generate relatively stable clusters by varying the parameter. The experimental comparison with the leading state-of-the-art alternatives demonstrates that the proposed algorithm outperforms DBSCAN and HDBSCAN in terms of affine invariance, and exceeds or matches the ro-bustness to noises of DBSCAN or HDBSCAN. The robust-ness to parameter selection is also demonstrated through the case study of clustering twitter data.

View Conference

Research Organization:: North Carolina State University, Raleigh, NC (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA), Office of Nonproliferation and Verification Research and Development (NA-22)

DOE Contract Number:: NA0002576

OSTI ID:: 1438413

Resource Relation:: Conference: 24. ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, San Francisco, CA (United States), 31 Oct- 3 Nov 2016

Country of Publication:: United States

Language:: English

References (26)

ST-DBSCAN: An algorithm for clustering spatial–temporal data Birant, Derya; Kut, Alp Data & Knowledge Engineering, Vol. 60, Issue 1 https://doi.org/10.1016/j.datak.2006.01.013	journal	January 2007
Locally Scaled Density Based Clustering Biçici, Ergun; Yuret, Deniz Adaptive and Natural Computing Algorithms: 8th International Conference, ICANNGA 2007, Warsaw, Poland, April 11-14, 2007, Proceedings, Part I https://doi.org/10.1007/978-3-540-71618-1_82	book	January 2007
Comparing clusterings—an information based distance Meilă, Marina Journal of Multivariate Analysis, Vol. 98, Issue 5 https://doi.org/10.1016/j.jmva.2006.11.013	journal	May 2007
KAZE Features Alcantarilla, Pablo Fernández; Bartoli, Adrien; Davison, Andrew J. Computer Vision – ECCV 2012, p. 214-227 https://doi.org/10.1007/978-3-642-33783-3_16	book	January 2012
Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey Bouwmans, Thierry; El Baf, Fida; Vachon, Bertrand Recent Patents on Computer Sciencee, Vol. 1, Issue 3 https://doi.org/10.2174/2213275910801030219	journal	November 2008
Hierarchical clustering schemes Johnson, Stephen C. Psychometrika, Vol. 32, Issue 3 https://doi.org/10.1007/BF02289588	journal	September 1967
Cluster analysis and display of genome-wide expression patterns Eisen, M. B.; Spellman, P. T.; Brown, P. O. Proceedings of the National Academy of Sciences, Vol. 95, Issue 25 https://doi.org/10.1073/pnas.95.25.14863	journal	December 1998
Ward’s Hierarchical Agglomerative Clustering Method: Which Algorithms Implement Ward’s Criterion? Murtagh, Fionn; Legendre, Pierre Journal of Classification, Vol. 31, Issue 3 https://doi.org/10.1007/s00357-014-9161-z	journal	October 2014
Characterizing the shapes of noisy, non-uniform, and disconnected point clusters in the plane Zhong, Xu; Duckham, Matt Computers, Environment and Urban Systems, Vol. 57 https://doi.org/10.1016/j.compenvurbsys.2016.01.003	journal	May 2016
Interval estimates of weighted effect sizes in the one-way heteroscedastic ANOVA Kulinskaya, E.; Staudte, R. G. British Journal of Mathematical and Statistical Psychology, Vol. 59, Issue 1 https://doi.org/10.1348/000711005X68174	journal	May 2006
A Quality Index Based on Data Depth and Multivariate Rank Tests Liu, Regina Y.; Singh, Kesar Journal of the American Statistical Association, Vol. 88, Issue 421 https://doi.org/10.1080/01621459.1993.10594317	journal	March 1993
OPTICS: ordering points to identify the clustering structure Ankerst, Mihael; Breunig, Markus M.; Kriegel, Hans-Peter Proceedings of the 1999 ACM SIGMOD international conference on Management of data - SIGMOD '99 https://doi.org/10.1145/304182.304187	conference	January 1999
Fast nonparametric classification based on data depth Lange, Tatjana; Mosler, Karl; Mozharovskyi, Pavlo Statistical Papers, Vol. 55, Issue 1 https://doi.org/10.1007/s00362-012-0488-4	journal	November 2012
An improved sampling-based DBSCAN for large spatial databases Borah, B.; Bhattacharyya, D. K. International Conference on Intelligent Sensing and Information Processing, 2004. Proceedings of https://doi.org/10.1109/ICISIP.2004.1287631	conference	January 2004
Automated Hierarchical Density Shaving: A Robust Automated Clustering and Visualization Framework for Large Biological Data Sets Gupta, Gunjan; Liu, Alexander; Ghosh, Joydeep IEEE/ACM Transactions on Computational Biology and Bioinformatics, Vol. 7, Issue 2 https://doi.org/10.1109/TCBB.2008.32	journal	April 2010
Model-Based Clustering, Discriminant Analysis, and Density Estimation Fraley, Chris; Raftery, Adrian E. Journal of the American Statistical Association, Vol. 97, Issue 458 https://doi.org/10.1198/016214502760047131	journal	June 2002
Comparing partitions Hubert, Lawrence; Arabie, Phipps Journal of Classification, Vol. 2, Issue 1 https://doi.org/10.1007/BF01908075	journal	December 1985
On the shape of a set of points in the plane Edelsbrunner, H.; Kirkpatrick, D.; Seidel, R. IEEE Transactions on Information Theory, Vol. 29, Issue 4 https://doi.org/10.1109/TIT.1983.1056714	journal	July 1983
Algorithm AS 136: A K-Means Clustering Algorithm Hartigan, J. A.; Wong, M. A. Applied Statistics, Vol. 28, Issue 1 https://doi.org/10.2307/2346830	journal	January 1979
General notions of statistical depth function Zuo, Yijun; Serfling, Robert The Annals of Statistics, Vol. 28, Issue 2 https://doi.org/10.1214/aos/1016218226	journal	April 2000
Classifying the segmentation of customer value via RFM model and RS theory Cheng, Ching-Hsue; Chen, You-Shyang Expert Systems with Applications, Vol. 36, Issue 3 https://doi.org/10.1016/j.eswa.2008.04.003	journal	April 2009
Choosing DBSCAN Parameters Automatically using Differential Evolution Karami, Amin; Johansson, Ronnie International Journal of Computer Applications, Vol. 91, Issue 7 https://doi.org/10.5120/15890-5059	journal	April 2014
CLARANS: a method for clustering objects for spatial data mining Ng, R. T. IEEE Transactions on Knowledge and Data Engineering, Vol. 14, Issue 5 https://doi.org/10.1109/TKDE.2002.1033770	journal	September 2002
Understanding Human Mobility from Twitter Jurdak, Raja; Zhao, Kun; Liu, Jiajun PLOS ONE, Vol. 10, Issue 7 https://doi.org/10.1371/journal.pone.0131469	journal	July 2015
An adaptive clustering algorithm for image segmentation Pappas, T. N. IEEE Transactions on Signal Processing, Vol. 40, Issue 4 https://doi.org/10.1109/78.127962	journal	April 1992
Clustering and classification based on the L1 data depth Jörnsten, Rebecka Journal of Multivariate Analysis, Vol. 90, Issue 1 https://doi.org/10.1016/j.jmva.2004.02.013	journal	July 2004

Similar Records

Data-driven cyber attack detection and mitigation for decentralized wide-area protection and control in smart grids

Other · Tue Jan 01 00:00:00 EST 2019 · OSTI ID:1985638

Wang, Pengyuan

Density-Aware Clustering Based on Aggregated Heat Kernel and Its Transformation

Journal Article · Mon Jun 01 00:00:00 EDT 2015 · ACM Transactions on Knowledge Discovery from Data · OSTI ID:1210172

Huang, Hao; Yoo, Shinjae; Yu, Dantong; +1 more

Automated identification of characteristic droplet size distributions in stratocumulus clouds utilizing a data clustering algorithm

Journal Article · Thu Jul 14 00:00:00 EDT 2022 · Artificial Intelligence for the Earth Systems · OSTI ID:1884127

Allwayin, Nithin; Larsen, Michael L.; Shaw, Alexander G.; +1 more

Related Subjects

58 GEOSCIENCES
97 MATHEMATICS AND COMPUTING
cluster analysis
data depth
density-based clustering analysis
affine invariant clustering

Data depth based clustering analysis

Citation Formats

References (26)

Similar Records

Related Subjects