{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T07:35:00Z","timestamp":1723016100853},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,7]]},"abstract":"Compression techniques for deep neural network models are becoming very important\n\nfor the efficient execution of high-performance deep learning systems on edge-computing devices.\n\nThe concept of model compression is also important for analyzing the generalization error of deep learning, known as the compression-based error bound.\n\nHowever, there is still huge gap between a practically effective compression method and its rigorous background of statistical learning theory. To resolve this issue, we develop a new theoretical framework for model compression\n\nand propose a new pruning method called {\\it spectral pruning} based on this framework.\n\nWe define the ``degrees of freedom'' to quantify the intrinsic dimensionality of a model\n\nby using the eigenvalue distribution of the covariance matrix across the internal nodes\n\nand show that the compression ability is essentially controlled by this quantity.\n\nMoreover, we present a sharp generalization error bound of the compressed model\n\nand characterize the bias--variance tradeoff induced by the compression procedure.\n\nWe apply our method to several datasets to justify our theoretical analyses and show the superiority of the the proposed method.<\/jats:p>","DOI":"10.24963\/ijcai.2020\/393","type":"proceedings-article","created":{"date-parts":[[2020,7,8]],"date-time":"2020-07-08T08:12:10Z","timestamp":1594195930000},"page":"2839-2846","source":"Crossref","is-referenced-by-count":5,"title":["Spectral Pruning: Compressing Deep Neural Networks via Spectral Analysis and its Generalization Error"],"prefix":"10.24963","author":[{"given":"Taiji","family":"Suzuki","sequence":"first","affiliation":[{"name":"The University of Tokyo"},{"name":"Center for Advanced Intelligence Project, RIKEN"}]},{"given":"Hiroshi","family":"Abe","sequence":"additional","affiliation":[{"name":"iPride Co., Ltd."}]},{"given":"Tomoya","family":"Murata","sequence":"additional","affiliation":[{"name":"NTT DATA Mathematical Systems Inc."}]},{"given":"Shingo","family":"Horiuchi","sequence":"additional","affiliation":[{"name":"NTT DATA Corporation"}]},{"given":"Kotaro","family":"Ito","sequence":"additional","affiliation":[{"name":"NTT DATA Mathematical Systems Inc."}]},{"given":"Tokuma","family":"Wachi","sequence":"additional","affiliation":[{"name":"NTT DATA Corporation"}]},{"given":"So","family":"Hirai","sequence":"additional","affiliation":[{"name":"NTT DATA Corporation"}]},{"given":"Masatoshi","family":"Yukishima","sequence":"additional","affiliation":[{"name":"NTT DATA Mathematical Systems Inc."}]},{"given":"Tomoaki","family":"Nishimura","sequence":"additional","affiliation":[{"name":"NTT DATA Corporation"}]}],"member":"10584","event":{"number":"28","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"acronym":"IJCAI-PRICAI-2020","name":"Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence {IJCAI-PRICAI-20}","start":{"date-parts":[[2020,7,11]]},"theme":"Artificial Intelligence","location":"Yokohama, Japan","end":{"date-parts":[[2020,7,17]]}},"container-title":["Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2020,7,8]],"date-time":"2020-07-08T22:14:53Z","timestamp":1594246493000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2020\/393"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2020,7]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2020\/393","relation":{},"subject":[],"published":{"date-parts":[[2020,7]]}}}