Evaluation of missing value imputation methods for wireless soil datasets | Personal and Ubiquitous Computing Skip to main content
Springer Nature Link
Log in

Evaluation of missing value imputation methods for wireless soil datasets

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Soil data are very important for hydrologists to model and predict the evolution of water–soil environments. In present, the soil data are often collected by unattended wireless sensing system and then inevitably involves continuous missing values due to the unreliability of system, which is different from the manually collected datasets with the data losses being sparsely distributed . This paper investigates seven typical methods that are used to infill soil missing data, and in particular we also attempt to employ the extreme learning machine in missing-data infilling. This work is aimed at answering such a question: Whether or not existing methods suit for wireless sensory soil dataset with continuous missing values, and how well they perform. With a real-world soil dataset involving complete samples as the benchmark, we evaluate and compare these methods , and analyze the possible reasons behind. This study provides insights for designing new methods that can effectively deal with the missing values in wireless sensory soil dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Budiman A, Fanany MI (2013) Pose-based 3d human motion analysis using extreme learning machine. In: 2013 IEEE 2nd global conference on consumer electronics (GCCE), pp 3–7

  2. Charoenhirunyingyosa S, Hondaa K, Kamthonkiatb D, Inesc A (2011) Soil moisture estimation from inverse modeling using multiple criteria functions. Comput Electron Agric 75(2):278–287

    Article  Google Scholar 

  3. Coopersmith E, Minsker B, Wenzel C, Gilmore B (2014) Machine learning assessments of soil drying for agricultural planning. Comput Electron Agric 104:93–104

    Article  Google Scholar 

  4. Culler D, Estrin D, Srivastava M (2004) Introduction: overview of sensor networks. Computer 37(8):41–49

    Article  Google Scholar 

  5. Dan L, Sun L, Dai W (2014) Wireless sensor networks system of forest habitat factors collection. J Harbin Inst Technol 46(7):123–128

    Google Scholar 

  6. Deo RC, Şahin M (2015) Application of the extreme learning machine algorithm for the prediction of monthly effective drought index in eastern Australia. Atmos Res 153:512–525

    Article  Google Scholar 

  7. Dumedah G, Coulibaly P (2011) Evaluation of statistical methods for infilling missing values in high-resolution soil moisture data. J Hydrol 400:95–102

    Article  Google Scholar 

  8. Dumedah G, Walker J, Chik L (2014) Assessing artificial neural networks and statistical methods for infilling missing soil moisture records. J Hydrol 515(16):330–344

    Article  Google Scholar 

  9. Farhangfar A, Kurgan L, Dy J (2008) Impact of imputation of missing values on classification error for discrete data. Pattern Recogn 41(12):3692–3705

    Article  MATH  Google Scholar 

  10. Gong J, Geng J, Chen Z (2015) Real-time GIS data model and sensor web service platform for environmental data management. Int J Health Geograph 14(2):1–13

    Google Scholar 

  11. Han P, Wang P, Zhang S, Zhu D (2010) Drought forecasting based on the remote sensing data using arima models. Math Comput Model 51(11–12):1398–1403

    Article  Google Scholar 

  12. Hardy A, Barr S, Mills J, Miller P (2012) Characterising soil moisture in transport corridor environments using airborne lidar and casi data. Hydrol Process 26(13):1925–1936

    Article  Google Scholar 

  13. Hsu HH, Yang AC, Lu MD (2011) KNN-DTW based missing value imputation for microarray time series data. J Comput 6(3):418–425

    Google Scholar 

  14. Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of international joint conference on neural networks, vol 2, pp 985–990

  15. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Article  Google Scholar 

  16. Kohn R, Ansley C (1986) Estimation, prediction, and interpolation for arima models with missing data. J Am Stat Assoc 81(395):751–761

    Article  MathSciNet  MATH  Google Scholar 

  17. Kornelsen K, Coulibaly P (2014) Comparison of interpolation, statistical, and data-driven methods for imputation of missing values in a distributed soil moisture dataset. J Hydrol Eng 19(1):26–43

    Article  Google Scholar 

  18. Kurban T, Beşdok E (2009) A comparison of RBF neural network training algorithms for inertial sensor based terrain classification. Sensors 9(8):6312–6329

    Article  Google Scholar 

  19. Lee W, Alchanatis V, Yang C, Hirafuji M, Moshou D, Li C (2010) Sensing technologies for precision specialty crop production. Comput Electron Agric 74(1):2–33

    Article  Google Scholar 

  20. Li J, Gao H (2008) Survey on sensor network research. J Comput Res Develop 45(1):1–15

    MathSciNet  Google Scholar 

  21. Lindenmayer D, Likens G (2010) The science and application of ecological monitoring. Biol Conserv 143(6):1317–1328

    Article  Google Scholar 

  22. Lingras P, Zhong M, Sharma S (1970) Evolutionary regression and neural imputations of missing values. Stud Fuzziness Soft Comput 226:151–163

    Article  Google Scholar 

  23. Meijering E, Falk H (2012) A chronology of interpolation: from ancient astronomy to modern signal and image processing. Proc IEEE 90(3):319–342

    Article  Google Scholar 

  24. Mohammed AA, Minhas R, Wu QMJ, Sid-Ahmed MA (2011) Human face recognition based on multidimensional PCA and extreme learning machine. Pattern Recogn 44(44):2588–2597

    Article  MATH  Google Scholar 

  25. Moorthy K, Mohamad MS, Deris S (2014) A review on missing value imputation algorithms for microarray gene expression data. Curr Bioinform 9(1):18–22

    Article  Google Scholar 

  26. Mukhopadhyay S, Jiang J (eds) (2013) Wireless sensor networks and ecological monitoring (smart sensors, measurement and instrumentation). Springer, Berlin

    Google Scholar 

  27. Nemes A, Wosten J, Varallyay G, Bouma J (2006) Soil water balance scenario studies using predicted soil hydraulic parameters. Hydrol Process 20(5):1075–1094

    Article  Google Scholar 

  28. Neruda R, Kudová P (2005) Learning methods for radial basis function networks. Future Gener Comput Syst 21(7):1131–1142

    Article  MATH  Google Scholar 

  29. Ojha T, Misraa S, Raghuwanshib N (2015) Wireless sensor networks for agriculture: the state-of-the-art in practice and future challenges. Comput Electron Agric 118:66–84

    Article  Google Scholar 

  30. Pigott TD (2001) A review of methods for missing data. Educ Res Eval 7(4):353–383

    Article  Google Scholar 

  31. Pomati F, Jokela J, Simora M, Veronesi M, Ibelings B (2011) An automated platform for phytoplankton ecology and aquatic ecosystem monitoring. Environ Sci Technol 45(22):9658–9665

    Article  Google Scholar 

  32. Saaban A, Zainudin L, Bakar MNA (2014) On piecewise interpolation techniques for estimating solar radiation missing values in Kedah. J Immunol 160(6):2824–2830

    Google Scholar 

  33. Schneider A (2012) Monitoring land cover change in urban and peri-urban areas using dense time stacks of landsat satellite data and a data mining approach. Remote Sens Environ 124:689–704

    Article  Google Scholar 

  34. Schwenker F, Kestler HA, Palm G (2001) Three learning phases for radial-basis-function networks. Neural Netw 14(4–5):439–458

    Article  MATH  Google Scholar 

  35. Shi-Chang D, Li-Feng X, Jian-Jun S (2006) Distributed sensor system for fault detection and isolation in multistage manufacturing systems. Int J Comput Appl Technol 25(4):1

    Google Scholar 

  36. Song Y, Crowcroft J, Zhang J (2012) Automatic epileptic seizure detection in EEGs based on optimized sample entropy and extreme learning machine. J Neurosci Methods 210(2):132–146

    Article  Google Scholar 

  37. Sultan Noman Qasem SMS (2011) Radial basis function network based on time variant multi-objective particle swarm optimization for medical diseases diagnosis. Appl Soft Comput 11(1):1427–1438

    Article  Google Scholar 

  38. Taormina R, Chau KW (2015) Data-driven input variable selection for rainfall-runoff modeling using binary-coded particle swarm optimization and extreme learning machines. J Hydrol 529:1617–1632

    Article  Google Scholar 

  39. Tsekouras GE, Tsimikas J (2013) On training RBF neural networks using input-output fuzzy clustering and particle swarm optimization. Fuzzy Sets Syst 221:65–89

    Article  MathSciNet  MATH  Google Scholar 

  40. Vachaud G, Silans APD, Balabanis P, Vauclin M (1985) Temporal stability of spatially measured soil water probability density function. Soil Sci Soc Am J 49(49):822–828

    Article  Google Scholar 

  41. Wang G, Garciab D, Liu Y, Jeua R, Dolmana A (2012) A three-dimensional gap filling method for large geophysical datasets: application to global satellite soil moisture observations. Eviron Modell Softw 30:139–142

    Article  Google Scholar 

  42. Wang J, Damevski K, Chen H (2015) Sensor data modeling and validating for wireless soil sensor network. Comput Electron Agric 112:75–82

    Article  Google Scholar 

  43. Wang N, Zhang N, Wang M (2006) Wireless sensors in agriculture and food industry\(\,^{\circ }\)™recent development and future perspective. Comput Electron Agric 50(1):1–14

    Article  Google Scholar 

  44. Yang J, Zhang C, Li X (2010) Integration of wireless sensor networks in environmental monitoring cyber infrastructure. Wireless Netw 16(4):1091–1108

    Article  Google Scholar 

  45. Yue L, Long M, Su KO (2014) Prediction of soil moisture based on extreme learning machine for an apple orchard. In: IEEE international conference on cloud computing and intelligence systems

Download references

Acknowledgements

This study was supported, in part, by the NSF of China with Grant No. 61300180 and by the Fundamental Research Funds for the Central Universities of China with Grant No. TD2014-01.

Author information

Authors and Affiliations

  1. Information School, Beijing Forestry University, 35 Tsinghua East Rd, Beijing, 100083, China

    Jia Shao & Wei Meng

  2. Rm 109, West Main Building, Information School, Beijing Forestry University, 35 Tsinghua East Rd, Beijing, 100083, China

    Guodong Sun

Authors
  1. Jia Shao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Wei Meng
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Guodong Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Wei Meng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shao, J., Meng, W. & Sun, G. Evaluation of missing value imputation methods for wireless soil datasets. Pers Ubiquit Comput 21, 113–123 (2017). https://doi.org/10.1007/s00779-016-0978-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-016-0978-9

Keywords