Abstract
To an increasing degree, data is a driving force for digitization, and hence also a key asset for numerous companies. In many businesses, various sources of data exist, which are isolated from one another in different domains, across a heterogeneous application landscape. Well-known centralized solution technologies, such as data warehouses and data lakes, exist to integrate data into one system, but they do not always scale well. Therefore, robust and decentralized ways to manage data can provide the companies with better value give companies a competitive edge over a single central repository. In this paper, we address why and when a monolithic data storage should be decentralized for improved scalability, and how to perform the decentralization. The paper is based on industrial experiences and the findings show empirically the potential of a distributed system as well as pinpoint the core pieces that are needed for its central management.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Cohen, J., Dolan, B., Dunlap, M., Hellerstein, J.M., Welton, C.: Mad skills: new analysis practices for big data. Proc. VLDB Endow. 2(2), 1481–1492 (2009)
Dehghani, Z.: How to move beyond a monolithic data lake to a distributed data mesh (2019). Martin Fowler’s blog. https://martinfowler.com/articles/data-monolith-to-mesh.html
Evans, E.: Domain-Driven Design: Tackling Complexity in the Heart of Software. Addison-Wesley Professional, Boston (2003)
Fitzgerald, B., Stol, K.J.: Continuous software engineering: a roadmap and agenda. J. Syst. Softw. 123, 176–189 (2017)
Hasselbring, W., Steinacker, G.: Microservice architectures for scalability, agility and reliability in e-commerce. In: 2017 IEEE International Conference on Software Architecture Workshops (ICSAW), pp. 243–246. IEEE (2017)
Kalske, M., Mäkitalo, N., Mikkonen, T.: Challenges when moving from monolith to microservice architecture. In: Garrigós, I., Wimmer, M. (eds.) ICWE 2017. LNCS, vol. 10544, pp. 32–47. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-74433-9_3
Kimball, R., Ross, M.: The Data Warehouse Toolkit. Wiley Computer Publishing, New York (2002)
Miloslavskaya, N., Tolstoy, A.: Big data, fast data and data lake concepts. Procedia Comput. Sci. 88, 300–305 (2016)
Nadareishvili, I., Mitra, R., McLarty, M., Amundsen, M.: Microservice Architecture: Aligning Principles, Practices, and Culture. O’Reilly Media, Inc., Sebastopol (2016)
Papazoglou, M.P., Traverso, P., Dustdar, S., Leymann, F.: Service-oriented computing: state of the art and research challenges. Computer 40(11), 38–45 (2007)
Perrey, R., Lycett, M.: Service-oriented architecture. In: Proceedings of 2003 Symposium on Applications and the Internet Workshops, pp. 116–119. IEEE (2003)
Stein, B., Morrison, A.: The enterprise data lake: better integration and deeper analytics. PwC Technol. Forecast Rethink. Integr. 1(1–9), 18 (2014)
ThoughtWorks: Data mesh (2020). https://www.thoughtworks.com/radar/techniques/data-mesh
Acknowledgement
This work is partly funded by Business Finland under grant agreement ITEA-2019-18022-IVVES and AIGA project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Loukiala, A., Joutsenlahti, JP., Raatikainen, M., Mikkonen, T., Lehtonen, T. (2021). Migrating from a Centralized Data Warehouse to a Decentralized Data Platform Architecture. In: Ardito, L., Jedlitschka, A., Morisio, M., Torchiano, M. (eds) Product-Focused Software Process Improvement. PROFES 2021. Lecture Notes in Computer Science(), vol 13126. Springer, Cham. https://doi.org/10.1007/978-3-030-91452-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-91452-3_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-91451-6
Online ISBN: 978-3-030-91452-3
eBook Packages: Computer ScienceComputer Science (R0)