Abstract
Insufficient amounts of historical data present a major challenge in real world supervised machine learning projects. Small and Medium-Sized Enterprises (SMEs) are particularly handicapped regarding the collection of historical data. A possible solution to this problem is data pooling, where data from different entities is combined to create larger datasets that are more suitable for supervised machine learning. In this study, we investigate the potential that data pooling has for six companies from the service industry located in Germany and Austria. We find that in the studied scenario each company can benefit from the other companies’ data under certain circumstances. In addition, while most companies benefit from a model that is trained with the data of all other companies, this is not always the case. This is because of specific business characteristics that can significantly affect datasets. In such a case, the key challenge is to determine which companies’ data to include in the pool, i.e., to define the pooling strategy. Therefore, we analyze all possible pooling strategies in our scenario and explain selected results with insights from data distribution and feature importance analysis. We conclude that the consideration of business and data characteristics is critical to the selection of an appropriate strategy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Note that since we only consider strategies where a company’s data is not split, we only look at a small subset of all possible pooling strategies. Moreover, we denote a strategy as “optimal”, if it achieves the best performance with respect to the subset of strategies we consider, which will most likely not be the global optimum.
References
Baltagi, B.H., Griffin, J.M.: Pooled estimators vs. their heterogeneous counterparts in the context of dynamic demand for gasoline. J. Econ. 77(2), 303–327 (1997)
Bolhuis, M., Rayner, B.: The More the Merrier? A Machine Learning Algorithm for Optimal Pooling of Panel Data (2020). https://doi.org/10.2139/ssrn.3583406
Borji, A.: Pros and cons of GAN evaluation measures. Comput. Vis. Image Underst. 179, 41–65 (2019)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Cuggia, M., Combes, S.: The French health data hub and the German medical informatics initiatives: two national projects to promote data sharing in healthcare (1) (2019). https://doi.org/10.1055/s-0039-1677917
Fuchs, B., Schumacher, A., Eggeling, E., Schlund, S.: Fraunhofer Austria KI-Studie: Künstliche Intelligenz in Österreichs Unternehmen (2022)
Louppe, G.: Understanding random forests: from theory to practice (2014). https://doi.org/10.13140/2.1.1570.5928
Hittmeir, M., Ekelhart, A., Mayer, R.: On the utility of synthetic data. In: Proceedings of the 14th International Conference on Availability, Reliability and Security, pp. 1–6. ACM, New York (2019). https://doi.org/10.1145/3339252.3339281
Hoogstrate, A.J., Palm, F.C., Pfann, G.A., Hoogstrate, A.J.: Pooling in dynamic panel-data models: an application to forecasting GDP growth rates. J. Bus. Econ. Stat. 18(3), 274 (2000)
Hulsen, T.: Sharing is caring-data sharing initiatives in healthcare. Int. J. Environ. Res. Public Health 17, 9 (2020)
IDSA: Sharing Data While Keeping Data Ownership. The Potential of IDS For The Data Economy (2018)
Lorenz, F.O., Simons, R.L., Conger, R.D., Elder, G.H., Johnson, C., Chao, W.: Married and recently divorced mothers’ stressful events and distress: tracing change across time. J. Marriage Fam. 59(1), 219 (1997)
McArdle, J.J., Hamagami, F., Meredith, W., Bradway, K.P.: Modeling the dynamic hypotheses of Gf–Gc theory using longitudinal life-span data. Learn. Individ. Differ. 12(1), 53–79 (2000)
McArdle, J.J., Prescott, C.A., Hamagami, F., Horn, J.L.: A contemporary method for developmental-genetic analyses of age changes in intellectual abilities. Dev. Neuropsychol. 14(1), 69–114 (1998)
Otto, B., et al.: Data ecosystems. Conceptual foundations, constituents and recommendations for action (2019)
Zhang, W., Deng, L., Zhang, L., Wu, D.: A survey on negative transfer. IEEE Trans. Neural Netw. Learn. Syst. (2021)
Zhuang, F., et al.: A comprehensive survey on transfer learning. Proc. IEEE 109(1), 43–76 (2020)
Acknowledgement
This research was supported by the Tyrolean provincial government and the Austrian Research Promotion Agency under the projects my Office ML (F.22742/18-2020) and Digital Innovation Hub West (873857 and F.17913/19-2020).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Czarnetzki, L., Kainz, F., Lächler, F., Laflamme, C., Bachlechner, D. (2022). Enabling Supervised Machine Learning Through Data Pooling: A Case Study with Small and Medium-Sized Enterprises in the Service Industry. In: Bergmann, R., Malburg, L., Rodermund, S.C., Timm, I.J. (eds) KI 2022: Advances in Artificial Intelligence. KI 2022. Lecture Notes in Computer Science(), vol 13404. Springer, Cham. https://doi.org/10.1007/978-3-031-15791-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-15791-2_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15790-5
Online ISBN: 978-3-031-15791-2
eBook Packages: Computer ScienceComputer Science (R0)