乐胖代购免代理版

{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T04:23:12Z","timestamp":1729138992984,"version":"3.27.0"},"reference-count":55,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2023,3,15]],"date-time":"2023-03-15T00:00:00Z","timestamp":1678838400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,15]],"date-time":"2023-03-15T00:00:00Z","timestamp":1678838400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["124020371"],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["01|18038A"],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Data Min Knowl Disc"],"published-print":{"date-parts":[[2023,5]]},"abstract":"Abstract<\/jats:title>Ensembles are among the state-of-the-art in many machine learning applications. With the ongoing integration of ML models into everyday life, e.g., in the form of the Internet of Things, the deployment and continuous application of models become more and more an important issue. Therefore, small models that offer good predictive performanceand<\/jats:italic>use small amounts of memory are required. Ensemble pruning is a standard technique for removing unnecessary classifiers from a large ensemble that reduces the overall resource consumption and sometimes improves the performance of the original ensemble. Similarly, leaf-refinement is a technique that improves the performance of a tree ensemble by jointly re-learning the probability estimates in the leaf nodes of the trees, thereby allowing for smaller ensembles while preserving their predictive performance. In this paper, we develop a new method that combines both approaches into a single algorithm. To do so, we introduce

$$L_1$$<\/jats:tex-math>L<\/mml:mi>1<\/mml:mn><\/mml:msub><\/mml:math><\/jats:alternatives><\/jats:inline-formula>regularization into the leaf-refinement objective, which allows us to jointly prune and refine trees at the same time. In an extensive experimental evaluation, we show that our approach not only offers statistically significantly better performance than the state-of-the-art but also offers a better accuracy-memory trade-off. We conclude our experimental evaluation with a case study showing the effectiveness of our method in a real-world setting.<\/jats:p>","DOI":"10.1007\/s10618-023-00921-z","type":"journal-article","created":{"date-parts":[[2023,3,15]],"date-time":"2023-03-15T18:02:38Z","timestamp":1678903358000},"page":"1230-1261","update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Joint leaf-refinement and ensemble pruning through $$L_1$$ regularization"],"prefix":"10.1007","volume":"37","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-2780-3618","authenticated-orcid":false,"given":"Sebastian","family":"Buschj\u00e4ger","sequence":"first","affiliation":[]},{"given":"Katharina","family":"Morik","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,3,15]]},"reference":[{"key":"921_CR1","doi-asserted-by":"crossref","unstructured":"Akash PS, Kadir M, Ali AA, Tawhid MNA, Shoyaib M (2019) Introducing confidence as a weight in random forest. In: 2019 international conference on robotics, electrical and signal processing techniques (ICREST). IEEE, pp 611\u2013616","DOI":"10.1109\/ICREST.2019.8644396"},{"key":"921_CR2","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1007\/978-3-319-14231-9_2","volume-title":"Decision-tree induction","author":"RC Barros","year":"2015","unstructured":"Barros RC, de Carvalho ACPLF, Freitas AA (2015) Decision-tree induction. Springer, Cham, pp 7\u201345. https:\/\/doi.org\/10.1007\/978-3-319-14231-9_2"},{"issue":"Apr","key":"921_CR3","first-page":"1063","volume":"13","author":"G Biau","year":"2012","unstructured":"Biau G (2012) Analysis of a random forests model. J Mach Learn Res 13(Apr):1063\u20131095","journal-title":"J Mach Learn Res"},{"issue":"2","key":"921_CR4","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1007\/s11749-016-0481-7","volume":"25","author":"G Biau","year":"2016","unstructured":"Biau G, Scornet E (2016) A random forest guided tour. TEST 25(2):197\u2013227","journal-title":"TEST"},{"issue":"11","key":"921_CR5","doi-asserted-by":"publisher","first-page":"1289","DOI":"10.3390\/electronics8111289","volume":"8","author":"S Branco","year":"2019","unstructured":"Branco S, Ferreira AG, Cabral J (2019) Machine learning in resource-scarce embedded systems, fpgas, and end-devices: a survey. Electronics 8(11):1289","journal-title":"Electronics"},{"issue":"2","key":"921_CR6","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1007\/BF00058655","volume":"24","author":"L Breiman","year":"1996","unstructured":"Breiman L (1996) Bagging predictors. Mach Learn 24(2):123\u2013140","journal-title":"Mach Learn"},{"key":"921_CR7","unstructured":"Breiman L (2000) Some infinity theory for predictor ensembles. Technical report, Technical Report 579, Statistics Dept. UCB"},{"key":"921_CR8","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324","author":"L Breiman","year":"2001","unstructured":"Breiman L (2001) Random forests. Mach Learn. https:\/\/doi.org\/10.1023\/A:1010933404324","journal-title":"Mach Learn"},{"key":"921_CR9","doi-asserted-by":"publisher","DOI":"10.1097\/IYC.0000000000000008","author":"G Brown","year":"2005","unstructured":"Brown G, Wyatt JL, Tino P (2005) Managing diversity in regression ensembles. JMLR. https:\/\/doi.org\/10.1097\/IYC.0000000000000008","journal-title":"JMLR"},{"issue":"1","key":"921_CR10","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1109\/TCSI.2017.2710627","volume":"65","author":"S Buschj\u00e4ger","year":"2017","unstructured":"Buschj\u00e4ger S, Morik K (2017) Decision tree and random forest implementations for fast filtering of sensor data. IEEE Trans Circuits Syst I Regul Pap 65(1):209\u2013222","journal-title":"IEEE Trans Circuits Syst I Regul Pap"},{"key":"921_CR11","unstructured":"Buschj\u00e4ger S, Morik K (2021) There is no double-descent in random forests. CoRR arXiv:2111.04409"},{"key":"921_CR12","doi-asserted-by":"publisher","unstructured":"Buschj\u00e4ger S, Chen K, Chen J, Morik K (2018) Realization of random forest for real-time evaluation through tree framing. In: ICDM, pp 19\u201328. https:\/\/doi.org\/10.1109\/ICDM.2018.00017","DOI":"10.1109\/ICDM.2018.00017"},{"key":"921_CR13","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1016\/j.patrec.2016.01.029","volume":"74","author":"GD Cavalcanti","year":"2016","unstructured":"Cavalcanti GD, Oliveira LS, Moura TJ, Carvalho GV (2016) Combining diversity measures for ensemble pruning. Pattern Recogn Lett 74:38\u201345","journal-title":"Pattern Recogn Lett"},{"issue":"7","key":"921_CR14","doi-asserted-by":"publisher","first-page":"5113","DOI":"10.1007\/s10462-020-09816-7","volume":"53","author":"T Choudhary","year":"2020","unstructured":"Choudhary T, Mishra V, Goswami A, Sarangapani J (2020) A comprehensive survey on model compression and acceleration. Artif Intell Rev 53(7):5113\u20135155","journal-title":"Artif Intell Rev"},{"key":"921_CR15","unstructured":"Cortes C, Mohri M, Syed U (2014) Deep boosting. In: Proceedings of the thirty-first international conference on machine learning (ICML 2014)"},{"key":"921_CR16","first-page":"1","volume":"7","author":"J Dem\u0161ar","year":"2006","unstructured":"Dem\u0161ar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1\u201330","journal-title":"J Mach Learn Res"},{"key":"921_CR17","unstructured":"Denil M, Matheson D, De\u00a0Freitas N (2014) Narrowing the gap: random forests in theory and in practice. In: International conference on machine learning (ICML)"},{"issue":"1","key":"921_CR18","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s10994-006-6226-1","volume":"63","author":"P Geurts","year":"2006","unstructured":"Geurts P, Ernst D, Wehenkel L (2006) Extremely randomized trees. Mach Learn 63(1):3\u201342","journal-title":"Mach Learn"},{"key":"921_CR19","doi-asserted-by":"crossref","unstructured":"Giacinto G, Roli F, Fumera G (2000) Design of effective multiple classifier systems by clustering of classifiers. In: Proceedings 15th international conference on pattern recognition. ICPR-2000, vol 2. IEEE, pp 160\u2013163","DOI":"10.1109\/ICPR.2000.906039"},{"key":"921_CR20","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1016\/j.neucom.2017.06.052","volume":"275","author":"H Guo","year":"2018","unstructured":"Guo H, Liu H, Li R, Wu C, Guo Y, Xu M (2018) Margin & diversity based ordering ensemble pruning. Neurocomputing 275:237\u2013246","journal-title":"Neurocomputing"},{"issue":"8","key":"921_CR21","doi-asserted-by":"publisher","first-page":"832","DOI":"10.1109\/34.709601","volume":"20","author":"TK Ho","year":"1998","unstructured":"Ho TK (1998) The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 20(8):832\u2013844","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"921_CR22","unstructured":"Jiang W, Nie F, Huang H (2015) Robust dictionary learning with capped l1-norm. In: Twenty-fourth international joint conference on artificial intelligence"},{"key":"921_CR23","doi-asserted-by":"crossref","unstructured":"Jiang Z, Liu H, Fu B, Wu Z (2017) Generalized ambiguity decompositions for classification with applications in active learning and unsupervised ensemble pruning. In: 31st AAAI conference on artificial intelligence, AAAI 2017, pp 2073\u20132079","DOI":"10.1609\/aaai.v31i1.10834"},{"key":"921_CR24","unstructured":"Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7\u20139, 2015, conference track proceedings. arXiv:1412.6980"},{"issue":"1","key":"921_CR25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1214\/aos\/1015362183","volume":"30","author":"V Koltchinskii","year":"2002","unstructured":"Koltchinskii V et al (2002) Empirical margin distributions and bounding the generalization error of combined classifiers. Ann Stat 30(1):1\u201350","journal-title":"Ann Stat"},{"key":"921_CR26","doi-asserted-by":"crossref","unstructured":"Kumar A, Sindhwani V (2015) Near-separable non-negative matrix factorization with l1 and Bregman loss functions. In: Proceedings of the 2015 SIAM international conference on data mining. SIAM, pp 343\u2013351","DOI":"10.1137\/1.9781611974010.39"},{"key":"921_CR27","unstructured":"Kumar A, Goyal S, Varma M (2017) Resource-efficient machine learning in 2 kb ram for the internet of things. In: International conference on machine learning. PMLR, pp 1935\u20131944"},{"key":"921_CR28","doi-asserted-by":"crossref","unstructured":"Lazarevic A, Obradovic Z (2001) Effective pruning of neural network classifier ensembles. In: IJCNN\u201901, vol 2. IEEE, pp 796\u2013801","DOI":"10.1109\/IJCNN.2001.939461"},{"key":"921_CR29","doi-asserted-by":"crossref","unstructured":"Li N, Yu Y, Zhou Z-H (2012) Diversity regularized ensemble pruning. In: ECML PKDD. Springer, pp 330\u2013345","DOI":"10.1007\/978-3-642-33460-3_27"},{"key":"921_CR30","unstructured":"Li H, Kadav A, Durdanovic I, Samet H, Graf HP (2016) Pruning filters for efficient convnets. arXiv:1608.08710"},{"issue":"6","key":"921_CR31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3136625","volume":"50","author":"J Li","year":"2017","unstructured":"Li J, Cheng K, Wang S, Morstatter F, Trevino RP, Tang J, Liu H (2017) Feature selection: A data perspective. ACM Comput Surv: CSUR 50(6):1\u201345","journal-title":"ACM Comput Surv: CSUR"},{"key":"921_CR32","doi-asserted-by":"crossref","unstructured":"Louppe G, Geurts P (2012) Ensembles on random patches. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 346\u2013361","DOI":"10.1007\/978-3-642-33460-3_28"},{"key":"921_CR33","doi-asserted-by":"crossref","unstructured":"Lu Z, Wu X, Zhu X, Bongard J (2010) Ensemble pruning via individual contribution ordering. In: Proceedings of the ACM SIGKDD, pp 871\u2013880","DOI":"10.1145\/1835804.1835914"},{"issue":"6","key":"921_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3205453","volume":"9","author":"C Lucchese","year":"2018","unstructured":"Lucchese C, Nardini FM, Orlando S, Perego R, Silvestri F, Trani S (2018) X-cleaver: learning ranking ensembles by growing and pruning trees. ACM Trans Intell Syst Technol: TIST 9(6):1\u201326","journal-title":"ACM Trans Intell Syst Technol: TIST"},{"key":"921_CR35","unstructured":"Margineantu DD, Dietterich TG (1997) Pruning adaptive boosting. In: ICML, vol 97, pp 211\u2013218"},{"key":"921_CR36","unstructured":"Mart\u00ednez-Mu\u00f1oz G, Su\u00e1rez A (2004) Aggregation ordering in bagging. In: Proceedings of the IASTED, pp 258\u2013263"},{"key":"921_CR37","doi-asserted-by":"crossref","unstructured":"Mart\u00ednez-Mu\u00f1oz G, Su\u00e1rez A (2006) Pruning in ordered bagging ensembles. In: ICML, pp 609\u2013616","DOI":"10.1145\/1143844.1143921"},{"issue":"2","key":"921_CR38","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1109\/TPAMI.2008.78","volume":"31","author":"G Mart\u00ednez-Mu\u00f1oz","year":"2008","unstructured":"Mart\u00ednez-Mu\u00f1oz G, Hern\u00e1ndez-Lobato D, Su\u00e1rez A (2008) An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Trans Pattern Anal Mach Intell 31(2):245\u2013259","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"921_CR39","unstructured":"Masoudinejad M, Ramachandran\u00a0Venkatapathy AK, Tondorf D, Heinrich D, Falkenberg R, Buschhoff M (2018) Machine learning based indoor localisation using environmental data in phynetlab warehouse. In: Smart SysTech 2018; European conference on smart objects, systems and technologies, pp 1\u20138"},{"key":"921_CR40","doi-asserted-by":"crossref","unstructured":"Oshiro TM, Perez PS, Baranauskas JA (2012) How many trees in a random forest? In: International workshop on machine learning and data mining in pattern recognition. Springer, pp 154\u2013168","DOI":"10.1007\/978-3-642-31537-4_13"},{"issue":"3","key":"921_CR41","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1561\/2400000003","volume":"1","author":"N Parikh","year":"2014","unstructured":"Parikh N, Boyd S (2014) Proximal algorithms. Found Trends Optim 1(3):127\u2013239","journal-title":"Found Trends Optim"},{"key":"921_CR42","unstructured":"Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, Kopf A, Yang E, DeVito Z, Raison M, Tejani A, Chilamkurthy S, Steiner B, Fang L, Bai J, Chintala S (2019) Pytorch: an imperative style, high-performance deep learning library. In: Advances in neural information processing systems 32, pp 8024\u20138035. http:\/\/papers.neurips.cc\/paper\/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf"},{"key":"921_CR43","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825\u20132830","journal-title":"J Mach Learn Res"},{"key":"921_CR44","unstructured":"Ravi KB, Serra J (2017) Cost-complexity pruning of random forests. arXiv:1703.05430"},{"key":"921_CR45","unstructured":"Ren S, Cao X, Wei Y, Sun J (2015) Global refinement of random forest. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 723\u2013730"},{"key":"921_CR46","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/8291.001.0001","volume-title":"Boosting: foundations and algorithms","author":"RE Schapire","year":"2012","unstructured":"Schapire RE, Freund Y (2012) Boosting: foundations and algorithms. The MIT Press, Cambridge"},{"key":"921_CR47","doi-asserted-by":"crossref","unstructured":"Shahhosseini M, Hu G (2020) Improved weighted random forest for classification problems. In: International online conference on intelligent decision science. Springer, pp 42\u201356","DOI":"10.1007\/978-3-030-66501-2_4"},{"key":"921_CR48","first-page":"100251","volume":"7","author":"M Shahhosseini","year":"2022","unstructured":"Shahhosseini M, Hu G, Pham H (2022) Optimizing ensemble weights and hyperparameters of machine learning models for regression problems. Mach Learn Appl 7:100251","journal-title":"Mach Learn Appl"},{"key":"921_CR49","unstructured":"Shotton J, Sharp T, Kohli P, Nowozin S, Winn J, Criminisi A (2013) Decision jungles: compact and rich models for classification. In: NIPS\u201913 proceedings of the 26th international conference on neural information processing systems, pp 234\u2013242"},{"issue":"1","key":"921_CR50","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","volume":"58","author":"R Tibshirani","year":"1996","unstructured":"Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B (Methodol) 58(1):267\u2013288","journal-title":"J R Stat Soc Ser B (Methodol)"},{"key":"921_CR51","doi-asserted-by":"crossref","unstructured":"Tsoumakas G, Partalas I, Vlahavas IP (2009) An ensemble pruning primer. In: Okun O, Valentini G (eds) Applications of supervised and unsupervised ensemble methods, Studies in computational intelligence, vol 245. Springer, pp 1\u201313","DOI":"10.1007\/978-3-642-03999-7_1"},{"issue":"Jul","key":"921_CR52","first-page":"1315","volume":"7","author":"Y Zhang","year":"2006","unstructured":"Zhang Y, Burer S, Street WN (2006) Ensemble pruning via semi-definite programming. J Mach Learn Res 7(Jul):1315\u20131338","journal-title":"J Mach Learn Res"},{"key":"921_CR53","doi-asserted-by":"publisher","DOI":"10.1201\/b12207","volume-title":"Ensemble methods: foundations and algorithms","author":"Z-H Zhou","year":"2012","unstructured":"Zhou Z-H (2012) Ensemble methods: foundations and algorithms. CRC Press, Boca Raton. https:\/\/doi.org\/10.1201\/b12207"},{"key":"921_CR54","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1007\/978-3-030-29859-3_25","volume-title":"Hybrid artificial intelligent systems","author":"P Zyblewski","year":"2019","unstructured":"Zyblewski P, Wo\u017aniak M (2019) Clustering-based ensemble pruning and multistage organization using diversity. In: P\u00e9rez Garc\u00eda H, S\u00e1nchez Gonz\u00e1lez L, Castej\u00f3n Limas M, Quinti\u00e1n Pardo H, Corchado Rodr\u00edguez E (eds) Hybrid artificial intelligent systems. Springer, Cham, pp 287-298"},{"issue":"3","key":"921_CR55","doi-asserted-by":"publisher","first-page":"1049","DOI":"10.1007\/s10044-020-00867-8","volume":"23","author":"P Zyblewski","year":"2020","unstructured":"Zyblewski P, Wo\u017aniak M (2020) Novel clustering-based pruning algorithms. Pattern Anal Appl 23(3):1049\u20131058","journal-title":"Pattern Anal Appl"}],"container-title":["Data Mining and Knowledge Discovery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-023-00921-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10618-023-00921-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-023-00921-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,16]],"date-time":"2024-10-16T13:51:36Z","timestamp":1729086696000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10618-023-00921-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,15]]},"references-count":55,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,5]]}},"alternative-id":["921"],"URL":"https:\/\/doi.org\/10.1007\/s10618-023-00921-z","relation":{},"ISSN":["1384-5810","1573-756X"],"issn-type":[{"type":"print","value":"1384-5810"},{"type":"electronic","value":"1573-756X"}],"subject":[],"published":{"date-parts":[[2023,3,15]]},"assertion":[{"value":"10 December 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 January 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}