An Adaptive Data-Driven Imputation Model for Incomplete Event Series | SpringerLink
Skip to main content

An Adaptive Data-Driven Imputation Model for Incomplete Event Series

  • Conference paper
  • First Online:
Advanced Data Mining and Applications (ADMA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14176))

Included in the following conference series:

  • 1179 Accesses

Abstract

Event sequences play as a general fine-grained representation for temporal asynchronous event streams. However, in practice, event sequences are often fragmentary and incomplete with censored intervals or missing data, making it hard for downstream prediction and decision-making tasks. In this work, we propose a fresh extension on the definition of the temporal point process, which conventionally characterizes chronological prediction based on historical events, and introduce inverse point process that characterizes counter-chronological attribution based on future events. These two point process models allow us to impute missing events for one partially observed sequence with conditional intensities in two symmetric directions. We further design a peer imitation learning algorithm that lets two models cooperatively learn from each other, leveraging imputed sequences given by the counterpart as the supervised signal. The training process consists of iterative learning of two models and facilitates them to achieve a consensus. We conduct extensive experiments on both synthetic and real-world datasets, which demonstrate that our model can recover incomplete event sequences very close to the ground-truth, with averagely 49.40% improvement compared with related competitors measured by normalized optimal transport distance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11210
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14013
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Antoniou, A.: Digital Signal Processing. McGraw-Hill, New York (2016)

    Google Scholar 

  2. Crites, R.H., Barto, A.G.: Improving elevator performance using reinforcement learning. In: NeurIPS, pp. 1017–1023 (1995)

    Google Scholar 

  3. Daley, D.J., Vere-Jones, D.: An Introduction to the Theory of Point Processes: Volume II: General Theory and Structure. Springer, Heidelberg (2007). https://doi.org/10.1007/978-0-387-49835-5

  4. Du, N., Dai, H., Trivedi, R., Upadhyay, U., Gomez-Rodriguez, M., Song, L.: Recurrent marked temporal point processes: embedding event history to vector. In: SIGKDD, pp. 1555–1564 (2016)

    Google Scholar 

  5. Enguehard, J., Busbridge, D., Bozson, A., Woodcock, C., Hammerla, N.Y.: Neural temporal point processes for modelling electronic health records (2020)

    Google Scholar 

  6. Fan, Y., Xu, J., Shelton, C.R.: Importance sampling for continuous time bayesian networks. J. Mach. Learn. Res. 11(Aug), 2115–2140 (2010)

    MathSciNet  MATH  Google Scholar 

  7. Goodfellow, I.J., et al.: Generative adversarial nets. In: NeurIPS, pp. 2672–2680 (2014)

    Google Scholar 

  8. Hawkes, A.G.: Spectra of some self-exciting and mutually exciting point processes. Biometrika 58(1), 83–90 (1971)

    Article  MathSciNet  MATH  Google Scholar 

  9. Ho, J., Ermon, S.: Generative adversarial imitation learning. In: NeurIPS, pp. 4565–4573 (2016)

    Google Scholar 

  10. Isham, V., Westcott, M.: A self-correcting point process. Stochastic Process. Appl. 8(3), 335–347 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  11. Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  12. Lee, Y., Vo, T.V., Lim, K.W., Soh, H.: Z-transforms and its inference on partially observable point processes. In: IJCAI, pp. 2369–2375 (2018)

    Google Scholar 

  13. Li, S., Xiao, S., Zhu, S., Du, N., Xie, Y., Song, L.: Learning temporal point processes via reinforcement learning. In: NeurIPS, pp. 10781–10791 (2018)

    Google Scholar 

  14. Mei, H., Qin, G., Eisner, J.: Imputing missing events in continuous-time event streams. In: ICML, pp. 4475–4485 (2019)

    Google Scholar 

  15. Nodelman, U., Shelton, C.R., Koller, D.: Continuous time bayesian networks. arXiv preprint arXiv:1301.0591 (2012)

  16. Pan, Z., Huang, Z., Lian, D., Chen, E.: A variational point process model for social event sequences. In: AAAI, pp. 173–180 (2020)

    Google Scholar 

  17. Rao, V., Teh, Y.W.: MCMC for continuous-time discrete-state systems. In: NeurIPS, pp. 701–709 (2012)

    Google Scholar 

  18. Reinhart, A.: A review of self-exciting spatio-temporal point processes and their applications. Stat. Sci. 33(3), 299–318 (2018)

    MathSciNet  MATH  Google Scholar 

  19. Schaubel, D.E., Cai, J.: Multiple imputation methods for recurrent event data with missing event category. Can. J. Stat. 34(4), 677–692 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  20. Shelton, C.R., Qin, Z., Shetty, C.: Hawkes process inference with missing data. In: AAAI, pp. 6425–6432 (2015)

    Google Scholar 

  21. Stomakhin, A., Short, M.B., Bertozzi, A.L.: Reconstruction of missing data in social networks based on temporal patterns of interactions. Inverse Prob. 27(11), 115013 (2011)

    Article  MathSciNet  Google Scholar 

  22. Upadhyay, U., De, A., Rodriguez, M.G.: Deep reinforcement learning of marked temporal point processes. In: NeurIPS, pp. 3172–3182 (2018)

    Google Scholar 

  23. Whong, C.: Foiling nyc’s taxi trip data (2014)

    Google Scholar 

  24. Wu, Q., Zhang, Z., Gao, X., Yan, J., Chen, G.: Learning latent process from high-dimensional event sequences via efficient sampling. In: NeurIPS, pp. 3842–3851 (2019)

    Google Scholar 

  25. Xiao, S., Yan, J., Yang, X., Zha, H., Chu, S.M.: Modeling the intensity function of point process via recurrent neural networks. In: AAAI, pp. 1597–1603 (2017)

    Google Scholar 

  26. Zhao, Y., Jiang, H., Wang, X.: Minimum edit distance-based text matching algorithm. In: NLPKE, pp. 1–4 (2010)

    Google Scholar 

  27. Zhou, K., Zha, H., Song, L.: Learning triggering kernels for multi-dimensional hawkes processes. In: ICML, pp. 1301–1309 (2013)

    Google Scholar 

Download references

Acknowledgement

This work was supported by the National Key R&D Program of China [2020YFB1707900]; the National Natural Science Foundation of China [62272302, 62172276], and Shanghai Municipal Science and Technology Major Project [2021SHZDZX0102].

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xiaofeng Gao .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, J., Ye, H., Gao, X., Wu, F., Kong, L., Chen, G. (2023). An Adaptive Data-Driven Imputation Model for Incomplete Event Series. In: Yang, X., et al. Advanced Data Mining and Applications. ADMA 2023. Lecture Notes in Computer Science(), vol 14176. Springer, Cham. https://doi.org/10.1007/978-3-031-46661-8_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-46661-8_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-46660-1

  • Online ISBN: 978-3-031-46661-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics