Optimized Multi-Modular Services: Empowering Earth Observation Data Processing
Volume 35, Issue 2 (2024), pp. 363–378
Pub. online: 4 April 2024
Type: Research Article
Open Access
Open Access
Received
1 September 2023
1 September 2023
Accepted
1 March 2024
1 March 2024
Published
4 April 2024
4 April 2024
Abstract
The significance of earth observation data spans diverse fields and domains, driving the need for efficient management. Nevertheless, the exponential increase in data volume brings new challenges that complicate processing and storing data. This article proposes an optimized multi-modular service for earth observation data management in response to these challenges. The suggested approach focuses on choosing the optimal configurations for the storage and processing layers to improve the performance and cost-effectiveness of managing data. By employing the recommended optimized strategies, earth observation data can be managed more effectively, resulting in fast data processing and reduced costs.
References
Asmaryan, S., Muradyan, V., Tepanosyan, G., Hovsepyan, A., Saghatelyan, A., Astsatryan, H., Grigoryan, H., Abrahamyan, R., Guigoz, Y., Giuliani, G. (2019). Paving the way towards an Armenian Data Cube. Data, 4(3), 117. https://doi.org/10.3390/data4030117.
Astsatryan, H., Hayrapetyan, A., Narsisian, W., Saribekyan, A., Asmaryan, Sh., Saghatelyan, A., Muradyan, V., Guigoz, Y., Giuliani, G. Ray, N. (2015a). An interoperable web portal for parallel geoprocessing of satellite image vegetation indices. Earth Science Informatics, 8, 453–460. https://doi.org/10.1007/s12145-014-0165-3.
Astsatryan, H., Hayrapetyan, A., Narsisian, W., Asmaryan, S., Saghatelyan, A., Muradyan, V., Giuliani, G., Guigoz, Y., Ray, N. (2015b). An interoperable cloud-based scientific GATEWAY for NDVI time series analysis. Computer Standards & Interfaces, 41, 79–84. https://doi.org/10.1016/j.csi.2015.02.001.
Astsatryan, H., Sahakyan, V., Shoukourian, Y., Dongarra, J., Cros, P.-H., Dayde, M., Oster, P. (2015c). Strengthening compute and data intensive capacities of Armenia. In: 2015 14th RoEduNet International Conference – Networking in Education and Research (RoEduNet NER), pp. 28–33. https://doi.org/10.1109/RoEduNet.2015.7311823.
Astsatryan, H., Kocharyan, A., Hagimont, D., Lalayan, A. (2020). Performance optimization system for hadoop and spark frameworks. Cybernetics and Information Technologies, 20(6), 5–17. https://doi.org/10.2478/cait-2020-0056.
Astsatryan, H., Lalayan, A., Kocharyan, A., Hagimont, D. (2021). Performance-efficient recommendation and prediction service for Big Data frameworks focusing on data compression and in-memory data storage indicators. Scalable Computing: Practice and Experience, 22, 401–412. https://doi.org/10.12694/scpe.v22i4.1945.
Astsatryan, H., Lalayan, A., Giuliani, G. (2023). Scalable data processing platform for earth observation data repositories. Scalable Computing: Practice and Experience, 24(1), 35–44. https://doi.org/10.12694/scpe.v24i1.2041.
Baumann, P. (2010). Beyond rasters: introducing the new OGC web coverage service 2.0. In: 18th ACM SIGSPATIAL International Symposium on Advances in Geographic Information Systems, ACM-GIS 2010, pp. 320–329. https://doi.org/10.1145/1869790.1869835.
Buyya, R., Ranjan, R., Calheiros, R.N. (2009). Modeling and simulation of scalable Cloud computing environments and the CloudSim toolkit: challenges and opportunities. In: 2009 International Conference on High Performance Computing & Simulation, pp. 1–11. https://doi.org/10.1109/HPCSIM.2009.5192685.
Duplyakin, D., Ricci, R., Maricq, A., Wong, G., Duerig, J., Eide, E., Stoller, L., Hibler, M., Johnson, D., Webb, K., Akella, A., Wang, K., Ricart, G., Landweber, L., Elliott, C., Zink, M., Cecchet, E., Kar, S., Mishra, P. (2019). The design and operation of CloudLab. In: Proceedings of the USENIX Annual Technical Conference (ATC), pp. 1–14.
Giuliani, G., Chatenoux, B., De Bono, A., Rodila, D., Richard, J.-P., Allenbach, K., Dao, H., Peduzzi, P. (2017). Building an earth observations data cube: lessons learned from the Swiss Data Cube (SDC) on generating Analysis Ready Data (ARD). Big Earth Data, 1(1–2), 100–117. https://doi.org/10.1080/20964471.2017.1398903.
Giuliani, G., Camara, G., Killough, B., Minchin, S. (2019). Earth observation open science: enhancing reproducible science using data cubes. Data, 4, 147, https://doi.org/10.3390/data4040147.
Giuliani, G., Egger, E., Italiano, J., Poussin, C., Richard, J.-P., Chatenoux, B. (2020). Essential variables for environmental monitoring: what are the possible contributions of earth observation data cubes? Data, 5(4). https://doi.org/10.3390/data5040100.
Guo, H.-D., Zhang, L., Zhu, L.-W. (2015). Earth observation big data for climate change research. Advances in Climate Change Research, 6(2), 108–117. Special issue on advances in Future Earth research. https://doi.org/10.1016/j.accre.2015.09.007.
Guzinski, R., Kass, S., Huber, S., Bauer-Gottwein, P., Jensen, I.H., Naeimi, V., Doubkova, M., Walli, A., Tottrup, C. (2014). Enabling the use of earth observation data for integrated water resource management in Africa with the water observation and information system. Remote Sensing, 6(8), 7819–7839. https://doi.org/10.3390/rs6087819.
Keshavarz-Ghorabaee, M., Amiri, M., Zavadskas, E.K., Turskis, Z., Antucheviciene, J. (2018). Simultaneous Evaluation of Criteria and Alternatives (SECA) for multi-criteria decision-making. Informatica, 29(2), 265–280. https://doi.org/10.15388/Informatica.2018.167.
Lalayan, A.G. (2023). Data compression-aware performance analysis of dask and spark for earth observation data processing. Mathematical Problems of Computer Science, 59, 35–44. https://doi.org/10.51408/1963-0100.
Lalayan, A., Astsatryan, H., Giuliani, G. (2023). A multi-objective optimization service for enhancing performance and cost efficiency in earth observation data processing workflows. Baltic Journal of Modern Computing, 11(3), 420–434. https://doi.org/10.22364/bjmc.2023.11.3.05.
Lee, C., Gasster, S., Plaza, A., Chang, C.-I., Huang, B. (2011). Recent developments in high performance computing for remote sensing: a review. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 4, 508–527. https://doi.org/10.1109/JSTARS.2011.2162643.
Poniszewska-Marańda, A., Czechowska, E. (2021). Kubernetes cluster for automating software production environment. Sensors, 21(5), 1910. https://doi.org/doi.org/10.3390/s21051910.
Rivest, S., Bédard, Y., Proulx, M.-J., Nadeau, M., Hubert, F., Pastor, J. (2005). SOLAP technology: merging business intelligence with geospatial technology for interactive spatio-temporal exploration and analysis of data. ISPRS Journal of Photogrammetry and Remote Sensing, 60(1), 17–33. https://doi.org/10.1016/j.isprsjprs.2005.10.002.
Rizvi, S.R., Killough, B., Cherry, A., Gowda, S. (2018). Lessons learned and cost analysis of hosting a full stack Open Data Cube (ODC) application on the Amazon Web Services (AWS). In: IGARSS 2018 – 2018 IEEE International Geoscience and Remote Sensing Symposium, pp. 8643–8646. https://doi.org/10.1109/IGARSS.2018.8518084.
Rocklin, M. (2015). Dask: parallel computation with blocked algorithms and task scheduling. In: Python in Science Conference, pp. 126–132. https://doi.org/10.25080/Majora-7b98e3ed-013.
Sefraoui, O., Aissaoui, M., Eleuldj, M. (2012). OpenStack: toward an open-source solution for cloud computing. International Journal of Computer Applications, 55(3), 38–42. https://doi.org/10.5120/8738-2991.
Singh, S.K., Laari, P.B., Mustak, S., Srivastava, P.K., Szabó, S. (2018). Modelling of land use land cover change using earth observation data-sets of Tons River Basin, Madhya Pradesh, India. Geocarto International, 33(11), 1202–1222. https://doi.org/10.1080/10106049.2017.1343390.
Yu, J., Yang, H., Sun, H., Wang, Y., Chen, K., Yu, Y., You, Y. (2021). Realtime remote sensing image publishing system based on COG technology. In: 2021 28th International Conference on Geoinformatics, pp. 1–5. https://doi.org/10.1109/IEEECONF54055.2021.9687665.
Zhao, Q., Yu, L., Li, X., Peng, D., Zhang, Y., Gong, P. (2021). Progress and trends in the application of Google Earth and Google Earth Engine. Remote Sensing, 13(18), 3778. https://doi.org/10.3390/rs13183778.
Žižović, M.M., Albijanić, M., Jovanović, V., Žižović, M. (2019). A new method of multi-criteria analysis for evaluation and decision making by dominant criterion. Informatica, 30(4), 819–832. https://doi.org/10.15388/Informatica.2019.231.
Biographies
Lalayan Arthur
A. Lalayan is a researcher at the Institute for Informatics and Automation Problems (IIAP) of NAS RA. He received his bachelor’s degree and master’s degree in informatics and computer science from National Polytechnic University of Armenia in 2019 and 2021, respectively. In 2024, he received his PhD from the IIAP NAS RA. His research interests include EO data processing, distributed processing, analytics, and optimization.
Astsatryan Hrachya
H. Astsatryan studied mathematics and graduated from the Yerevan State University in 1998. In 2001, he received his PhD from the Institute for Informatics and Automation Problems of NAS RA and in 2020, a habilitation degree from the National Polytechnic Institute of Toulouse. His research experience is related to the topics of e-infrastructures and scientific computations. He has authored more than 80 articles in the revised journals and proceedings.
Poghosyan Suren
S. Poghosyan is a senior scientific researcher at the Institute for Informatics and Automation Problems of NAS RA. His primary research focus areas include distributed networks, cellular automata, self-organized criticality, information dissemination models, gossip/broadcast problems, and graph theory.
Giuliani Gregory
G. Giuliani is the head of the Digital Earth Unit and Swiss Data Cube Project Leader at GRID-Geneva of the United Nations Environment Programme and a senior lecturer at the University of Geneva’s Institute for Environmental Sciences. He is a geologist and environmental scientist who specializes in remote sensing, geographical information systems, and spatial data infrastructures. He also worked at GRID-Geneva of the United Nations Environment Programme since 2001, where he was previously the focal point for spatial data infrastructure and is currently the head of the Digital Earth Unit. Dr. Giuliani’s research focuses on land change science and how Earth observations can be used to monitor and assess environmental changes and support sustainable development.
