Abstract
Real-world time series often present missing values due to sensor malfunctions or human errors. Traditionally, missing values are simply omitted or reconstructed through imputation or interpolation methods. Omitting missing values may cause temporal discontinuity. Reconstruction methods, on the other hand, alter in some way the original time series. In this paper, we consider an application in the field of meteorological variables that exploits end-to-end machine learning. The idea is to entrust the task of dealing with missing values to a suitably trained recurrent neural network that completely by-passes the phase of reconstruction of missing values. A difficult case of reproduction of a rainfall field from five rain gauges in Northern Italy is used as an example, and the results are compared to those computed by more traditional methods. The proposed methodology is general-purpose and can be easily applied to every kind of spatial time series prediction problem, quite common in many environmental studies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Radi, N.F.A., Zakaria, R., Azman, M.A.Z.: Estimation of missing rainfall data using spatial interpolation and imputation methods. In: AIP Conference Proceedings, vol. 1643, no. 1, pp. 42–48. American Institute of Physics (2015)
Dhevi, A.S.: Imputing missing values using inverse distance weighted interpolation for time series data. In: 2014 Sixth International Conference on Advanced Computing (ICoAC), pp. 255–259. IEEE (2014)
Teegavarapu, R.S.: Missing precipitation data estimation using optimal proximity metric-based imputation, nearest-neighbour classification and cluster-based interpolation methods. Hydrol. Sci. J. 59(11), 2009–2026 (2014)
Cressie, N.: The origins of kriging. Math. Geol. 22(3), 239–252 (1990)
Stein, M.L.: Interpolation of Spatial Data: Some Theory for Kriging. Springer Science & Business Media, New York (2012)
Liu, Y., Weerts, A., Clark, M., Hendricks Franssen, H.J., Kumar, S., Moradkhani, H., Seo, D.J., Schwanenberg, D., Smith, P., Van Dijk, A.I.J.M., Van Velzen, N.: Advancing data assimilation in operational hydrologic forecasting: progresses, challenges, and emerging opportunities. Hydrol. Earth Syst. Sci. 16, 3863–3887 (2012)
Weerts, A.H., Seo, D.J., Werner, M., Schaake, J.: Operational hydrologic ensemble forecasting. In: Applied Uncertainty Analysis for Flood Risk Management, pp. 387–406 (2014)
Pezij, M., Augustijn, D.C., Hendriks, D.M., Hulscher, S.J.: The role of evidence-based information in regional operational water management in the Netherlands. Environ. Sci. Policy 93, 75–82 (2019)
Lagasio, M., Parodi, A., Pulvirenti, L., Meroni, A.N., Boni, G., Pierdicca, N., Marzano, F.S., Luini, L., Venuti, G., Realini, E., Gatti, A., Tagliaferro, G., Barindelli, S., Monti Guarnieri, A., Goga, K., Terzo, O., Rucci, A., Passera, E., Kranzlmueller, D., Rommen, B.: A synergistic use of a high-resolution numerical weather prediction model and high-resolution earth observation products to improve precipitation forecast. Remote Sensing 11, 2387 (2019)
Xingjian, S.H.I., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., Woo, W.C.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in Neural Information Processing Systems, pp. 802–810 (2015)
Ng, A.: Machine learning yearning, p. 96 (2017). http://www.mlyearning.org/
Sangiorgio, M., Barindelli, S.: Spatio-temporal analysis of intense convective storms tracks in a densely urbanized Italian basin. ISPRS Int. J. Geo-Inf. 9, 183 (2020)
Sangiorgio, M., Barindelli, S., Biondi, R., Solazzo, E., Realini, E., Venuti, G., Guariso, G.: Improved extreme rainfall events forecasting using neural networks and water vapor measures. In: Proceedings of the 6th International Conference on Time Series and Forecasting (ITISE), Granada, Spain, vol. 2, pp. 820–826 (2019)
Sangiorgio, M., Barindelli, S., Biondi, R., Solazzo, E., Realini, E., Venuti, G., Guariso, G.: A comparative study on machine learning techniques for intense convective rainfall events forecasting. In: Advances in Time Series and Forecasting (Contributions to Statistics) stage of publication (under review)
Foresti, L., Sideris, I.V., Panziera, L., Beusch, L., Nerini, D., Germann, U.: Nowcasting orographic precipitation growth and decay using machine learning algorithms on a 10-year radar archive in the Swiss Alps. In: EGU General Assembly Conference Abstracts, vol. 20, p. 6361 (2018)
Meeker, W.Q., Escobar, L.A.: Statistical Methods for Reliability Data. Wiley, New York (2014)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Goodfellow, I., Bengio, Y., Courville, A.: Deep learning. MIT Press, Cambridge (2016)
Acknowledgements
We would like to thank ARPA Lombardia for providing the rainfall dataset. Valerio Guglieri is supported by Fondazione Cariplo (2017-0725), project: Lombardy-based Advanced Meteorological Predictions and Observations (LAMPO). The authors would like to thank all the administrative, management and scientific partners of the project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sangiorgio, M., Barindelli, S., Guglieri, V., Venuti, G., Guariso, G. (2020). Reconstructing Environmental Variables with Missing Field Data via End-to-End Machine Learning. In: Iliadis, L., Angelov, P., Jayne, C., Pimenidis, E. (eds) Proceedings of the 21st EANN (Engineering Applications of Neural Networks) 2020 Conference. EANN 2020. Proceedings of the International Neural Networks Society, vol 2. Springer, Cham. https://doi.org/10.1007/978-3-030-48791-1_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-48791-1_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-48790-4
Online ISBN: 978-3-030-48791-1
eBook Packages: Computer ScienceComputer Science (R0)