Semantic segmentation of real-time sensor data stream for complex activity recognition | Personal and Ubiquitous Computing
Skip to main content
Springer Nature Link
Log in

Semantic segmentation of real-time sensor data stream for complex activity recognition

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Data segmentation plays a critical role in performing human activity recognition in the ambient assistant living systems. It is particularly important for complex activity recognition when the events occur in short bursts with attributes of multiple sub-tasks. Although substantial efforts have been made in segmenting the real-time sensor data stream such as static/dynamic window sizing approaches, little has been explored to exploit object semantic for discerning sensor data into multiple threads of activity of daily living. This paper proposes a semantic-based approach for segmenting sensor data series using ontologies to perform terminology box and assertion box reasoning, along with logical rules to infer whether the incoming sensor event is related to a given sequences of the activity. The proposed approach is illustrated using a use-case scenario which conducts semantic segmentation of a real-time sensor data stream to recognise an elderly persons complex activities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Awan MA, Guangbin Z, Kim S-D (2012) Activity recognition in WSN: A data-driven approach. Computing and Convergence Technology (ICCCT), 2012 7th International Conference on IEEE

  2. Azkune G, Almeida A, Lpez-de-Ipia D, Chen L (2015) Extending knowledge-driven activity models through data-driven learning techniques. Expert Syst Appl 42(6):3115–3128. doi:10.1016/j.eswa.2014.11.063

    Article  Google Scholar 

  3. BakhshandehAbkenar A, Loke SW (2014) MyActivity: cloud-hosted continuous activity recognition using ontology-based stream reasoning. In: MOBILECLOUD '14 Proceedings of the 2014 2nd IEEE International Conference on Mobile Cloud Computing, Services, and Engineering, pp 117–126. doi:10.1109/MobileCloud.2014.27

  4. Balduini M, Della Valle E, DellAglio D, Tsytsarau M, Palpanas T, Confalonieri C (2013) Social listening of city scale events using the streaming linked data framework. In: ISWC '13 Proceedings of the 12th International Semantic Web Conference - Part II 8219:1–16. doi:10.1007/978-3-642-41338-4_1

  5. Barnaghi P, Wang W, Dong L, Wang C (2013) A linked-data model for semantic sensor streams. In: GREENCOM-ITHINGS-CPSCOM '13 Proceedings of the 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing, pp 468–475. doi:10.1109/GreenCom-iThings-CPSCom.95

  6. Calbimonte J, Jeung H, Corcho O, Aberer K (2011) Semantic sensor data search in a large-scale federated sensor network. Semant Sens Netw 839:23–38

    Google Scholar 

  7. Chen L, Hoey J, Nugent CD, Cook DJ, Yu Z (2012) Sensor-based activity recognition. IEEE Trans Syst Man Cybern Part C Appl Rev 42:790–808. doi:10.1109/TSMCC.2012.2198883

    Article  Google Scholar 

  8. Chen L, Nugent CD, Wang H (2012) A knowledge-driven approach to activity recognition in smart homes. IEEE Trans Knowl Data Eng 24:961–974. doi:10.1109/TKDE.2011.51

    Article  Google Scholar 

  9. Chen L, Nugent C, Okeyo G (2014) An ontology-based hybrid approach to activity modeling for smart homes. IEEE Trans Hum Mach Syst 44:92–105. doi:10.1109/THMS.2013.2293714

    Article  Google Scholar 

  10. Cook D, Feuz K, Krishnan N (2013) Transfer learning for activity recognition: a survey. Knowl Inf Syst 36:537–556. doi:10.1007/s10115-013-0665-3

    Article  Google Scholar 

  11. Ferroni G, Bonfigli R, Principi E, Squartini S, Piazza F (2015) A deep neural network approach for voice activity detection in multi-room domestic scenarios. Neural Netw. doi:10.1109/IJCNN.2015.7280510

    Google Scholar 

  12. Fobel A, Subramanian N (2016) Comparison of the performance of Drools and Jena rule-based systems for event processing on the semantic web. In: 2016 IEEE/ACIS 14th international conference on software engineering, research, management and applications SERA 2016, pp 24–30. doi:10.1109/SERA.2016.7516153

  13. Ganz F, Barnaghi P, Carrez F (2014) Acquis From Sens Data 10:1–12

    Google Scholar 

  14. Gu T, Wang L, Wu Z, Tao X, Lu J (2011) A pattern mining approach to sensor-based human activity recognition. IEEE Trans Knowl Data Eng 23:1359–1372. doi:10.1109/TKDE.2010.184

    Article  Google Scholar 

  15. Hu DH, Yang Q (2008) CIGAR: concurrent and interleaving goal and activity recognition. In: AAAI conference on artificial intelligence, pp 1363–1368

  16. Huang V, Javed MK (2008) Semantic sensor information description and processing. In: Proc-2nd Int Conf Sens Technol Appl, SENSORCOMM 2008, Incl MESH 2008 Conf Mesh Networks; ENOPT 2008 Energy Optim Wirel Sensors Networks, UNWAT 2008 Under Water Sensors Syst, pp 456–461. doi:10.1109/SENSORCOMM.2008.23

  17. Kang H, Hebert M, Efros AA, Kanade T (2015) Data-driven objectness input image database of object regions high objectness regions. IEEE Trans Pattern Anal Mach Intell 37:189–195

    Article  Google Scholar 

  18. Khan JA, Kumar S (2015) OWL, RDF, RDFS inference derivation using Jena semantic framework and pellet reasoner. In: 2014 international conference on advances in engineering and technology research ICAETR 2014, p 07. doi:10.1109/ICAETR.2014.7012871

  19. Kharlamov E, Kotidis Y, Mailis T, Neuenstadt C, Nikolaou C, zcep,Svingos C, Zheleznyakov D, Lamparter S, Horrocks I, Ioannidis Y, Mller R (2016) Towards analytics aware ontology based access to static and streaming data (extended version), p 122. doi:10.1007/978-3-319-46547-0

  20. Kuka C, Nicklas D (2014) Enriching sensor data processing with quality semantics. In: PERCOM Work 2014 IEEE international conference on pervasive computing and communication workshops, pp 437–442. doi:10.1109/PerComW.2014.6815246

  21. Liu Y, Nie L, Liu L, Rosenblum DS (2015) From action to activity: sensor-based activity recognition. Neurocomputing. doi:10.1016/j.neucom.2015.08.096

    Google Scholar 

  22. Okeyo G, Chen L, Wang H (2014) Combining ontological and temporal formalisms for composite activity modelling and recognition in smart homes. Futur Gener Comput Syst 39:29–43. doi:10.1016/j.future.2014.02.014

    Article  Google Scholar 

  23. Okeyo G, Chen L, Wang H, Sterritt R (2014) Dynamic sensor data segmentation for real time activity recognition. Pervasive Mob Comput 10(1):155–172

    Article  Google Scholar 

  24. Okeyo G, Chen L, Wang H, Sterritt R (2014) Dynamic sensor data segmentation for real-time knowledge-driven activity recognition. Pervasive Mob Comput 10:155–172. doi:10.1016/j.pmcj.2012.11.004

    Article  Google Scholar 

  25. Ramar K, Mirnalinee TT (2012) An ontological representation for Tsunami early warning system. In: Advances in engineering, science and management (ICAESM), 2012 international conference on IEEE, pp 93–98

  26. Riboni D, Bettini C (2011) OWL 2 modeling and reasoning with complex human activities. Pervasive Mob Comput 7:379–395. doi:10.1016/j.pmcj.2011.02.001

    Article  Google Scholar 

  27. Snchez D, Tentori M, Favela J (2008) Activity recognition for the smart hospital. IEEE Intell Syst 23:50–57. doi:10.1109/MIS.2008.18

    Article  Google Scholar 

  28. Triboan D, Chen L, Chen F (2016) Towards a mobile assistive system using service- oriented architecture. In (2016) IEEE Symp. Syst. Eng. Towar. IEEE, Oxford, Serv, pp 187–196

  29. Triboan D, Chen L, Chen F, Wang Z (2016) Towards a service-oriented architecture for a mobile assistive system with real-time environmental sensing. Tsinghua Sci Technol 21:581–597

    Article  Google Scholar 

  30. Valle ED, Grossniklaus M (2010) C-SPARQL: a continuous query language for rdf data streams. Int J Semant Comput 4:3–25

    Article  MATH  Google Scholar 

  31. W3C (2004) SWRL: a semantic web rule language combining OWL and RuleML. https://www.w3.org/Submission/SWRL/. Accessed 20 Aug 2016

  32. W3C (2005) Semantic sensor network ontology. https://www.w3.org/2005/Incubator/ssn/ssnx/ssn. Accessed 1 Oct 2016

  33. W3C (2011) SPIN-overview and motivation. https://www.w3.org/Submission/spin-overview/. Accessed 22 Aug 2016

  34. W3C (2014) RDF stream processors implementation-rdf stream processing community group. https://www.w3.org/community/rsp/wiki/RDF_Stream_Processors_Implementation. Accessed 5 Oct 2016

  35. Wan J, OGrady MJ, OHare GMP (2015) Dynamic sensor event segmentation for real-time activity recognition in a smart home context. Pers Ubiquitous Comput 19:287301. doi:10.1007/s00779-014-0824-x

    Article  Google Scholar 

  36. Wei W, Barnaghi P (2009) Semantic annotation and reasoning for sensor data. In: Barnaghi P, Moessner K, Presser M, Meissner S (eds) Lect. Notes Comput. Sci. Springer, Berlin, pp 66–76

  37. Ye J, Stevenson G, Dobson S (2014) USMART: an unsupervised semantic mining activity recognition technique. In: ACM transactions on intelligent systems, pp 4:16–116:27. doi:10.1145/2662870

  38. Ye J, Stevenson G, Dobson S (2014) KCAR: a knowledge-driven approach for concurrent activity recognition. Pervasive Mob Comput 19:4770. doi:10.1016/j.pmcj.2014.02.003

    Google Scholar 

  39. Zhang S, Guo J, Yu Z, Lei C, Mao C, Wang H (2010) An approach of domain ontology construction based on resource model and Jena. In: 2010 third international symposium on information processing, pp 311–315. doi:10.1109/ISIP.2010.44

  40. Zhang W, Duan L, Chen J (2010) Reasoning and realization based on ontology model and Jena. In: proceedings of the 2010 IEEE 5th international conference on bio-inspired computing: theories and applications BIC-TA 2010, pp 1057–1060. doi:10.1109/BICTA.2010.5645115

  41. Zep L, Mller R, Neuenstadt C (2014) A stream-temporal query language for ontology based data access. KI 2014 Adv Artif Intell, pp 183–194. doi:10.1007/978-3-319-11206-0_18

Download references

Author information

Authors and Affiliations

  1. Context, Intelligence and Interaction Research Group (CIIRG), De Montfort University, Leicester, UK

    Darpan Triboan, Liming Chen & Feng Chen

  2. Department of Information Engineering, Dalian University, Dalian, China

    Zumin Wang

Authors
  1. Darpan Triboan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zumin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zumin Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Triboan, D., Chen, L., Chen, F. et al. Semantic segmentation of real-time sensor data stream for complex activity recognition. Pers Ubiquit Comput 21, 411–425 (2017). https://doi.org/10.1007/s00779-017-1005-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-017-1005-5

Keywords

Search

Navigation