Abstract
Filling large data-gaps in Micro-Meteorological data has mostly been done using interpolation techniques based on a marginal distribution sampling. Those methods work well but need a large horizon of the previous events to achieve good results since they do not model the system but only rely on previously encountered iterations. In this paper, we propose to use multi-head deep attention networks to fill gaps in Micro-Meteorological Data. This methodology couples large-scale information extraction with modeling capabilities that cannot be achieved by interpolation-like techniques. Unlike Bidirectional RNNs, our architecture is not recurrent, it is simple to tune and our data efficiency is higher. We apply our architecture to real-life data and clearly show its applicability in agriculture, furthermore, we show that it could be used to solve related problems such as filling gaps in cyclic-multivariate-time-series.
***.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Please note that this is only true in attention heads. We use a mask in the overall structure to copy the non-gap-points.
- 2.
- 3.
Data available upon request.
References
Abadi, M., Agarwal, A., Barham, P., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). http://tensorflow.org/. software available from tensorflow.org
Che, Z., Purushotham, S., Cho, K., Sontag, D., Liu, Y.: Recurrent neural networks for multivariate time series with missing values. Sci. Rep. 8(1), 6085 (2018)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Coutinho, E.R., Silva, R.M.d., Madeira, J.G.F., Coutinho, P.R.d.O.d., Boloy, R.A.M., Delgado, A.R.S., et al.: Application of artificial neural networks (ANNs) in the gap filling of meteorological time series. Revista Brasileira de Meteorologia 33(2), 317–328 (2018)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Falge, E., et al.: Gap filling strategies for long term energy flux data sets. Agric. For. Meteorol. 107(1), 71–77 (2001)
Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.N.: Convolutional sequence to sequence learning. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 1243–1252. JMLR. org (2017)
Graves, A., Fernández, S., Schmidhuber, J.: Bidirectional LSTM networks for improved phoneme classification and recognition. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 799–804. Springer, Heidelberg (2005). https://doi.org/10.1007/11550907_126
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Lange, M., Dechant, B., Rebmann, C., Vohland, M., Cuntz, M., Doktor, D.: Validating MODIS and sentinel-2 NDVI products at a temperate deciduous forest site using two independent ground-based sensors. Sensors 17(8), 1855 (2017)
Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol. 30, p. 3 (2013)
Moffat, A.M., et al.: Comprehensive comparison of gap-filling techniques for eddy covariance net carbon fluxes. Agric. For. Meteorol. 147(3–4), 209–232 (2007)
Papale, D., Valentini, R.: A new assessment of European forests carbon exchanges by eddy fluxes and artificial neural network spatialization. Glob. Change Biol. 9(4), 525–535 (2003)
Reichstein, M., et al.: On the separation of net ecosystem exchange into assimilation and ecosystem respiration: review and improved algorithm. Glob. Change Biol. 11(9), 1424–1439 (2005)
Richard, A., Mahé, A., Pradalier, C., Rozenstein, O., Geist, M.: A comprehensive benchmark of neural networks for system identification (2019)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wutzler, T., et al.: Basic and extensible post-processing of eddy covariance flux data with REddyProc. Biogeosciences 15(16), 5015–5030 (2018)
Acknowledgements
We would like to thank Lionel Clavien from our partner InnoBoost SA in Switzerland for providing us access to the server used for our experiments as well as some starting help on the platform. This work was supported by a grant from the Ministry of Science and Technology (MOST), Israel, under the France-Israel Maimonide Program, & Ministry of Europe and Foreign Affairs (MEAE), and the Ministry of Higher Education, Research and Innovation (MESRI) of France.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Richard, A., Fine, L., Rozenstein, O., Tanny, J., Geist, M., Pradalier, C. (2021). Filling Gaps in Micro-meteorological Data. In: Dong, Y., Ifrim, G., Mladenić, D., Saunders, C., Van Hoecke, S. (eds) Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track. ECML PKDD 2020. Lecture Notes in Computer Science(), vol 12461. Springer, Cham. https://doi.org/10.1007/978-3-030-67670-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-67670-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67669-8
Online ISBN: 978-3-030-67670-4
eBook Packages: Computer ScienceComputer Science (R0)
