{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,4,9]],"date-time":"2025-04-09T21:03:05Z","timestamp":1744232585441,"version":"3.37.3"},"reference-count":36,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2015,3,2]],"date-time":"2015-03-02T00:00:00Z","timestamp":1425254400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"the Internatioanal Collaboration Key Program of the CAS","award":["171111KYSB20130002"]},{"name":"the Strategic Priority Research Program of the CAS","award":["XDA06010403"]},{"DOI":"10.13039\/501100001809","name":"NSF of China","doi-asserted-by":"crossref","award":["61100163, 61133004, 61222204, 61221062, 61333014, 61321491, and 61303158"],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"the 863 Program of China","award":["2012AA012202 and 2012AA010902"]},{"name":"the DARPA PERFECT program"},{"name":"the 10000 Talent Program"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Des. Autom. Electron. Syst."],"published-print":{"date-parts":[[2015,3,2]]},"abstract":"Architectural design spaces of microprocessors are often exponentially large with respect to the pending processor parameters. To avoid simulating all configurations in the design space, machine learning and statistical techniques have been utilized to build regression models for characterizing the relationship between architectural configurations and responses (e.g., performance or power consumption). However, this article shows that the accuracy variability of many learning techniques over different design spaces and benchmarks can be significant enough to mislead the decision-making. This clearly indicates a high risk of applying techniques that work well on previous modeling tasks (each involving a design space, benchmark, and design objective) to a new task, due to which the powerful tools might be impractical.<\/jats:p>\n Inspired by ensemble learning in the machine learning domain, we propose a robust framework called ELSE to reduce the accuracy variability of design space modeling. Rather than employing a single learning technique as in previous investigations, ELSE employs distinct learning techniques to build multiple base regression models for each modeling task. This is not a trivial combination of different techniques (e.g., always trusting the regression model with the smallest error). Instead, ELSE carefully maintains the diversity of base regression models and constructs a metamodel from the base models that can provide accurate predictions even when the base models are far from accurate. Consequently, we are able to reduce the number of cases in which the final prediction errors are unacceptably large. Experimental results validate the robustness of ELSE: compared with the widely used artificial neural network over 52 distinct modeling tasks, ELSE reduces the accuracy variability by about 62%. Moreover, ELSE reduces the average prediction error by 27% and 85% for the investigated MIPS and POWER design spaces, respectively.<\/jats:p>","DOI":"10.1145\/2668118","type":"journal-article","created":{"date-parts":[[2015,3,3]],"date-time":"2015-03-03T14:08:19Z","timestamp":1425391699000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Robust Design Space Modeling"],"prefix":"10.1145","volume":"20","author":[{"given":"Qi","family":"Guo","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA"}]},{"given":"Tianshi","family":"Chen","sequence":"additional","affiliation":[{"name":"Chinese Academy of Sciences, Beijing, China"}]},{"given":"Zhi-Hua","family":"Zhou","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}]},{"given":"Olivier","family":"Temam","sequence":"additional","affiliation":[{"name":"Inria, France"}]},{"given":"Ling","family":"Li","sequence":"additional","affiliation":[{"name":"Chinese Academy of Sciences, Beijing, China"}]},{"given":"Depei","family":"Qian","sequence":"additional","affiliation":[{"name":"Beihang University, Beijing, China"}]},{"given":"Yunji","family":"Chen","sequence":"additional","affiliation":[{"name":"Chinese Academy of Sciences, Beijing, China"}]}],"member":"320","published-online":{"date-parts":[[2015,3,2]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1815961.1815967"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/339647.339657"},{"key":"e_1_2_1_3_1","unstructured":"A. Buja and W. Stuetzle. 2000. The effect of bagging on variance bias and mean squared error. Tech. rep. AT&T Labs-Research. http:\/\/www-stat.wharton.upenn.edu\/∼∼buja\/PAPERS\/paper-ustats.pdf. A. Buja and W. Stuetzle. 2000. The effect of bagging on variance bias and mean squared error. Tech. rep. AT&T Labs-Research. http:\/\/www-stat.wharton.upenn.edu\/∼∼buja\/PAPERS\/paper-ustats.pdf."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.2007.21"},{"key":"e_1_2_1_5_1","first-page":"97","article-title":"Machine-learning research: Four current directions","volume":"18","author":"Dietterich T. G.","year":"1998","unstructured":"T. G. Dietterich . 1998 . Machine-learning research: Four current directions . AI Mag. 18 , 97 -- 136 . T. G. Dietterich. 1998. Machine-learning research: Four current directions. AI Mag. 18, 97--136.","journal-title":"AI Mag."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.2007.26"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1450095.1450103"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1016218223"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/2283516.2283674"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/MM.2011.77"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1168857.1168882"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.2006.6"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/1299042.1299059"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.982906"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1839667.1839670"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1168857.1168881"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.2008.4771797"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1654059.1654116"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2514641.2514647"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/40.768496"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.614"},{"volume-title":"Proceedings of the 1st Workshop on Statistical and Machine Learning Approaches Applied to Architectures and Compilation (SMART'07)","author":"Ould-Ahmed-Vall E.","key":"e_1_2_1_22_1","unstructured":"E. Ould-Ahmed-Vall , J. Woodlee , C. Yount , and K. A. Doshi . 2007a. On the comparison of regression algorithms for computer architecture performance analysis of software applications . In Proceedings of the 1st Workshop on Statistical and Machine Learning Approaches Applied to Architectures and Compilation (SMART'07) . E. Ould-Ahmed-Vall, J. Woodlee, C. Yount, and K. A. Doshi. 2007a. On the comparison of regression algorithms for computer architecture performance analysis of software applications. In Proceedings of the 1st Workshop on Statistical and Machine Learning Approaches Applied to Architectures and Compilation (SMART'07)."},{"volume-title":"Proceedings of International Symposium on Performance Analysis of Systems and Software (ISPASS'07)","author":"Ould-Ahmed-Vall E.","key":"e_1_2_1_23_1","unstructured":"E. Ould-Ahmed-Vall , J. Woodlee , C. Yount , K. A. Doshi , and S. Abraham . 2007b. Using model trees for computer architecture performance analysis of software applications . In Proceedings of International Symposium on Performance Analysis of Systems and Software (ISPASS'07) . 116--125. E. Ould-Ahmed-Vall, J. Woodlee, C. Yount, K. A. Doshi, and S. Abraham. 2007b. Using model trees for computer architecture performance analysis of software applications. In Proceedings of International Symposium on Performance Analysis of Systems and Software (ISPASS'07). 116--125."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2009.2028681"},{"volume-title":"Proceedings of the Design, Automation and Test in Europe Conference (DATE'13)","author":"Paone E.","key":"e_1_2_1_25_1","unstructured":"E. Paone , N. Vahabi , V. Zaccaria , C. Silvano , D. Melpignano , G. Haugou , and T. Lepley . 2013. Improving simulation speed and accuracy for many-core embedded platforms with ensemble models . In Proceedings of the Design, Automation and Test in Europe Conference (DATE'13) . E. Paone, N. Vahabi, V. Zaccaria, C. Silvano, D. Melpignano, G. Haugou, and T. Lepley. 2013. Improving simulation speed and accuracy for many-core embedded platforms with ensemble models. In Proceedings of the Design, Automation and Test in Europe Conference (DATE'13)."},{"key":"e_1_2_1_26_1","unstructured":"J. Renau B. Fraguela J. Tuck W. Liu M. Prvulovic L. Ceze S. Sarangi P. Sack K. Strauss and P. Montesinos. 2005. SESC simulator. http:\/\/sesc.sourceforge.net. J. Renau B. Fraguela J. Tuck W. Liu M. Prvulovic L. Ceze S. Sarangi P. Sack K. Strauss and P. Montesinos. 2005. SESC simulator. http:\/\/sesc.sourceforge.net."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022648800760"},{"key":"e_1_2_1_28_1","unstructured":"D. Tarjan S. Thoziyoor and N. P. Jouppi. 2006. Cacti 4.0. Tech. rep. HPL-2006-86. D. Tarjan S. Thoziyoor and N. P. Jouppi. 2006. Cacti 4.0. Tech. rep. HPL-2006-86."},{"volume-title":"The Nature of Statistical Learning Theory","author":"Vapnik V. N.","key":"e_1_2_1_29_1","unstructured":"V. N. Vapnik . 1995. The Nature of Statistical Learning Theory . Springer . V. N. Vapnik. 1995. The Nature of Statistical Learning Theory. Springer."},{"volume-title":"Proceedings of the European Conference on Machine Learning (ECML'97)","author":"Wang Y.","key":"e_1_2_1_30_1","unstructured":"Y. Wang and I. H. Witten . 1997. Induction of model trees for predicting continuous classes . In Proceedings of the European Conference on Machine Learning (ECML'97) . Y. Wang and I. H. Witten. 1997. Induction of model trees for predicting continuous classes. In Proceedings of the European Conference on Machine Learning (ECML'97)."},{"key":"e_1_2_1_31_1","volume-title":"Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations","author":"Witten I. H.","year":"2005","unstructured":"I. H. Witten and E. Frank . 2005 . Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations 2 nd Ed. Morgan Kaufmann , San Fransisco . I. H. Witten and E. Frank. 2005. Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations 2nd Ed. Morgan Kaufmann, San Fransisco.","edition":"2"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1996.8.7.1341"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/4235.585893"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80023-1"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2006.404"},{"volume-title":"Ensemble Methods: Foundations and Algorithms","author":"Zhou Z.-H.","key":"e_1_2_1_36_1","unstructured":"Z.-H. Zhou . 2012. Ensemble Methods: Foundations and Algorithms . Chapman and Hall\/CRC , Boca Raton, FL . Z.-H. Zhou. 2012. Ensemble Methods: Foundations and Algorithms. Chapman and Hall\/CRC, Boca Raton, FL."}],"container-title":["ACM Transactions on Design Automation of Electronic Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2668118","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,30]],"date-time":"2022-12-30T19:32:26Z","timestamp":1672428746000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2668118"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,3,2]]},"references-count":36,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2015,3,2]]}},"alternative-id":["10.1145\/2668118"],"URL":"https:\/\/doi.org\/10.1145\/2668118","relation":{},"ISSN":["1084-4309","1557-7309"],"issn-type":[{"type":"print","value":"1084-4309"},{"type":"electronic","value":"1557-7309"}],"subject":[],"published":{"date-parts":[[2015,3,2]]},"assertion":[{"value":"2014-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-03-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}