Data recovery algorithm based on generative adversarial networks in crowd sensing Internet of Things | Personal and Ubiquitous Computing Skip to main content
Springer Nature Link
Log in

Data recovery algorithm based on generative adversarial networks in crowd sensing Internet of Things

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Internet of Things has developed quickly to share data from billions of physical devices. Completeness of data is important especially in crowd sensing Internet of Things. How to recover the lost data is a fundamental operation to utilize the coming of Internet of Things. Existing data recovery algorithms depend heavy on the accuracy distribution of environmental data and result in bad performance when reconstructing the lost data. This paper introduces a data recovery algorithm based on generative adversarial networks. The convolution neural network is used as the basic model of this algorithm. We add a restore network to reload the unlost data after recovery in this algorithm. The algorithm mainly includes two parts: (1) training process, in which all the collected sensory data are used to train the proposed generative adversarial networks model and (2) data recovery process, in which the lost data is recovered by using the trained generator. We use random loss dataset and periodic loss dataset to validate the data recovery performance. Finally, these two cases can verify that the recovery algorithm based on generative adversarial network is more enhanced compared with the comparison experiment under the three metrics of mean square error, mean absolute error, and R-square. The results show that our proposed algorithm can obtain data that are reliable and thus improve the performance of data recovery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Data is available on kaggle dataset. https://www.kaggle.com/nphantawee/pump-sensor-data

Code availability

All codes used during the study are available from the corresponding author by request.

References

  1. Gubbi J, Buyya R, Marusic S, Palaniswami M (2013) Internet of Things (IoT): a vision, architectural elements, and future directions. Futur Gener Comput Syst 29(7):1645–1660

  2. M. Zanjreh, H. Larijani (2015) A survey on centralised and distributed clustering routing algorithms for WSNs. IEEE 81st Vehicular Technology Conference, 2015: 1–6

  3. Zhou Y, Wang N, Xiang W (2016) Clustering hierarchy protocol in wireless sensor networks using an improved PSO algorithm. IEEE Access 5:2241–2253

    Article  Google Scholar 

  4. Wu M, Tan L, Xiong N (2015) Data prediction, compression, and recovery in clustered wireless sensor networks for environmental monitoring applications. Inf Sci 329:800–818

    Article  Google Scholar 

  5. Song W, Huang R, Xu M, Shirazi B, Lahusen R (2010) Design and deployment of sensor network for real-time high-fidelity volcano monitoring. IEEE Transactions on Parallel and Distributed Systems 21(11):1658–1674

    Article  Google Scholar 

  6. L. Mo, Y. He, Y. Liu, J. Zhao, S. Tang, X. Li (2009) Canopy closure estimates with GreenOrbs: sustainable sensing in the forest. ACM conference on embedded networked sensor systems, 99–112

  7. Giri P, Ng K, Phillips W (2019) Wireless sensor network system for landslide monitoring and warning. IEEE Trans Instrum Meas 68(4):1210–1220

    Article  Google Scholar 

  8. Rong H, Wang Z, Jiang H, Xiao Z, Zeng F (2019) Energy-aware clustering and routing in infrastructure failure areas with D2D communication. IEEE Internet Things J 6(5):8645–8657

    Article  Google Scholar 

  9. Mao Z, Su Y, Xu G, Wang X, Huang Y, Yue W, Sun L, Xiong N (2019) Spatio-temporal deep learning method for ADHD fMRI classification. Inf Sci 499:1–11

    Article  Google Scholar 

  10. Bellavista P, Belli D, Chessa S, Foschini L (2019) A social-driven edge computing architecture for mobile crowd sensing management. IEEE Commun Mag 57(4):68–73

    Article  Google Scholar 

  11. Cheng H, Wu L, Li R, Huang F, Tu C, Yu Z (2019) Data recovery in wireless sensor networks based on attribute correlation and extremely randomized trees. J Ambient Intell Humaniz Comput 23:1–15

    Google Scholar 

  12. Zou Z, Li Z, Shen S, Wang R (2016) Energy-efficient data recovery via greedy algorithm for wireless sensor networks. International Journal of Distributed Sensor Networks 2016:1–9

    Google Scholar 

  13. He J, Sun G, Zhang Y, Wang Z (2015) Data recovery in wireless sensor networks with joint matrix completion and sparsity constraints. IEEE Commun Lett 19(12):2230–2233

    Article  Google Scholar 

  14. Yu Z, Zhang D, Wang Z, Guo B, Roussaki I, Doolin K, Claffey E (2017) Toward context-aware mobile social networks. IEEE Commun Mag 55(10):168–175

    Article  Google Scholar 

  15. Diallo O, Rodrigues J, Sene M, Lloret J (2015) Distributed database management techniques for wireless sensor networks. IEEE Transactions on Parallel and Distributed Systems 26(2):604–620

    Article  Google Scholar 

  16. Guo W, Zhu W, Yu Z, Wang J, Guo B (2019) A survey of task allocation: contrastive perspectives from wireless sensor networks and mobile crowdsensing. IEEE Access 99:1–1

    Google Scholar 

  17. Huang M, Liu W, Wang T, Song H, Li X, Liu A (2019) A queuing delay utilization scheme for on-path service aggregation in services oriented computing networks. IEEE Access 7(1):23816–23833

    Article  Google Scholar 

  18. Xie X, Liu X, Qi H, Xiao B, Li K, Wu J (2019) Geographical correlation-based data collection for sensor-augmented RFID systems. IEEE Trans Mob Comput 99:1–1

    Google Scholar 

  19. Bishop CM (2006) Pattern recognition and machine learning. Springer. isbn:978-0-387-31073-2

  20. J. Koza, F. Bennett, D. Andre, M. Keane (1996) Automated design of both the topology and sizing of analog electrical circuits using genetic programming. Artificial Intelligence in Design ‘96, 151–170

  21. Hong Y, Hwang U, Yoo J, Yoon S (2017) How generative adversarial networks and its variants work: an overview of GAN. ACM Comput Surv 52(1):1–41

    Article  Google Scholar 

  22. Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory 52(4):1289–1306

    Article  MathSciNet  MATH  Google Scholar 

  23. Liao L, Li K, Yang C, Liu J (2019) Low-cost image compressive sensing with multiple measurement rates for object detection. Sensors 19(9):2079

    Article  Google Scholar 

  24. Quer G, Masiero R, Pillonetto G, Rossi M, Zorzi M (2012) Sensing, compression, and recovery for WSNs: sparse signal modeling and monitoring framework. IEEE Trans Wirel Commun 11(10):3447–3461

    Article  Google Scholar 

  25. Masiero R, Quer G, Munaretto D, Rossi M, Widmer J, Zorzi M (2009) Data acquisition through joint compressive sensing and principal component analysis. IEEE Global Telecommunications Conference:1–6

  26. Quer G, Masiero R, Munaretto D, Rossi M, Zorzi M (2009) On the interplay between routing and signal representation for compressive sensing in wireless sensor networks. Information Theory and Applications Workshop:206–215

  27. C. Luo, F. Wu, J. Sun, C. W. Chen (2009) Compressive data gathering for large-scale wireless sensor networks. International Conference on Mobile Computing and Networking, 145–156

  28. Singh D, Goudar R, Pant B, Rao S (2014) Cluster head selection by randomness with data recovery in WSN. CSI Transactions on ICT 2(2):97–107

    Article  Google Scholar 

  29. G. Chen, X. Liu, L. Kong, J. Liu, W. Shu, M. Wu (2013) JSSDR: joint-sparse sensory data recovery in wireless sensor networks. 2013 IEEE 9th International Conference on Wireless and Mobile Computing, Networking and Communications, 367–374

  30. Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Transaction on Information Theory 13(1):21–27

    Article  MATH  Google Scholar 

  31. Magán-Carrión R, Camacho J, García-Teodoro P (2015) Multivariate statistical approach for anomaly detection and lost data recovery in wireless sensor networks. International Journal of Distributed Sensor Networks 2015:1–20

    Google Scholar 

  32. Cheng H, Su Z, Xiong N, Xiao Y (2016) Energy-efficient node scheduling algorithms for wireless sensor networks. Inf Sci 329:461–477

    Article  MATH  Google Scholar 

  33. Cheng H, Wu L, Zhang Y, Zhang Y, Xiong N (2018) Data recovery in wireless sensor networks using Markov random field model. Tenth International Conference on Advanced Computational Intelligence (ICACI) 2018:706–711

  34. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial networks. Adv Neural Inf Proces Syst 3:2672–2680

    Google Scholar 

  35. A. Radford, L. Metz, S. Chintala (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. Computer Science

    Google Scholar 

  36. D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, A. Efros (2016) Context encoders: feature learning by inpainting. IEEE Conference on Computer Vision and Pattern Recognition

    Google Scholar 

  37. Y. Li, S. Liu, J. Yang, M. Yang (2017) Generative face completion. IEEE Conference on Computer Vision and Pattern Recognition

    Book  Google Scholar 

  38. C. Esteban, S. L. Hyland, G. Ratsch. Real-valued (medical): time series generation with recurrent conditional GANs. arXiv preprint arXiv:1706.02633, 2017

  39. D. Li, D. Chen, B. Jin, L. Shi, J. Goh, S.-K. Ng (2019) MAD-GAN: multivariate anomaly detection for time series data with generative adversarial networks, International Conference on Artificial Neural Networks, 703–716

  40. Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Proces Syst 25(2):1097–1105

    Google Scholar 

  41. Pump sensor data (2019-03-04) https://www.kaggle.com/nphantawee/pump-sensor-data

  42. Keković G, Sekulić S (2019) Detection of change points in time series with moving average filters and wavelet transform application to EEG signals. Neurophysiology 51(1):2–8

    Article  Google Scholar 

  43. Bulac C, Bulac A (2016) Decision trees. The Institute of Electrical and Electronics Engineers:819–844

  44. Iizuka S, Simo-Serra E, Ishikawa H (2017) Globally and locally consistent image completion. ACM Trans Graph 36(4):1–14

    Article  Google Scholar 

Download references

Funding

This work is supported in part by the Science Foundation of Fujian Province of China under Grand No. 2019J01245.

Author information

Authors and Affiliations

  1. College of Mathematics and Computer Science, Fuzhou University, Fuzhou, 350116, Fujian, China

    Yushi Shi, Xiaoqi Zhang, Qiaohong Hu & Hongju Cheng

  2. Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing, Fuzhou, 350116, Fujian, China

    Yushi Shi, Xiaoqi Zhang, Qiaohong Hu & Hongju Cheng

Authors
  1. Yushi Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoqi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiaohong Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongju Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongju Cheng.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, Y., Zhang, X., Hu, Q. et al. Data recovery algorithm based on generative adversarial networks in crowd sensing Internet of Things. Pers Ubiquit Comput 27, 537–550 (2023). https://doi.org/10.1007/s00779-020-01428-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-020-01428-w

Keywords

Search

Navigation