Discovering activity patterns in office environment using a network of low-resolution visual sensors | Journal of Ambient Intelligence and Humanized Computing
Skip to main content
Springer Nature Link
Log in

Discovering activity patterns in office environment using a network of low-resolution visual sensors

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Understanding activity patterns in office environments is important in order to increase workers’ comfort and productivity. This paper proposes an automated system for discovering activity patterns of multiple persons in a work environment using a network of cheap low-resolution visual sensors (900 pixels). Firstly, the users’ locations are obtained from a robust people tracker based on recursive maximum likelihood principles. Secondly, based on the users’ mobility tracks, the high density positions are found using a bivariate kernel density estimation. Then, the hotspots are detected using a confidence region estimation. Thirdly, we analyze the individual’s tracks to find the starting and ending hotspots. The starting and ending hotspots form an observation sequence, where the user’s presence and absence are detected using three powerful Probabilistic Graphical Models (PGMs). We describe two approaches to identify the user’s status: a single model approach and a two-model mining approach. We evaluate both approaches on video sequences captured in a real work environment, where the persons’ daily routines are recorded over 5 months. We show how the second approach achieves a better performance than the first approach. Routines dominating the entire group’s activities are identified with a methodology based on the Latent Dirichlet Allocation topic model. We also detect routines which are characteristic of persons. More specifically, we perform various analysis to determine regions with high variations, which may correspond to specific events.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Abbreviations

\(\left( x',y'\right)\) :

Tracked position

\(f'\) :

Probability density function of tracked positions

B :

Kernel function in \(f'\)

\(\mathbf {H}\) :

Bandwidth matrix in \(f'\)

\(({x'}^{(m)},{y'}^{(m)})\) :

Position in cluster m

m :

Cluster index

L :

Number of clusters

\(K^{(m)}\) :

Number of positions in cluster m

\(\mathbf {U}^{(m)}\) :

Positions matrix

\(\mathbf {C}^{(m)}\) :

Covariance matrix of clustered positions

\(\lambda _{1}^{(m)}\), \(\lambda _{2}^{(m)}\) :

Eigenvalues of the covariance matrix \(\mathbf {C}^{(m)}\)

\(\rho\) :

Correlation coefficient

\(\sigma _{{x'}^{(m)}}\) :

Standard deviation of clustered \({x'}^{(m)}\) position

\(\sigma _{{y'}^{(m)}}\) :

Standard deviation of clustered \({y'}^{(m)}\) position

A :

Ellipse confidence level

\(\mathbf {V}^{(m)}\) :

Eigenvectors matrix

\(\mathbf {D}^{(m)}\) :

Diagonal matrix

\(\theta ^{(m)}\) :

Rotation angel of the ellipse

S :

Start of hotspot

E :

End of hotspot

T :

Track with start hotspot S and end hotspot E

I :

Number of positions in track T

\((g_{x}^{(m)},g_{y}^{(m)})\) :

Center of hotspot (cluster) m

F :

Hotspot threshold

\(\mathbf {x}_t\) :

Feature vector at time instant t

\(\mathbf {x}_{1:T}\) :

Sequence of observations

\(\mathbf {y}_{1:T}\) :

Sequence of states

T :

Number of time steps

\(\hat{N}\) :

Number of persons

Q :

Number of states

\(p(\mathbf {x}_{1:T},y)\) :

Joint distribution of \(\mathbf {x}_{1:T}\) and y

\(\gamma\), \(\delta\) :

Potential functions

\(\epsilon\) :

Actual potential

f :

Feature function

\(p(\mathbf {y} \mid \mathbf {x})\) :

Conditional probability of \(\mathbf {y}\) given \(\mathbf {x}\)

\(Z_{x}\) :

Normalization function

N :

Pattern length

e :

Word in a document (presence status)

d :

Document of words (presence status sequence)

z :

Topic (activity pattern)

K :

Number of topics

G :

Number of constructed words

M :

Number of documents

V :

Vocabulary of words

\(\theta\) :

Matrix of day-specific mixture weights

\(\Phi\) :

Matrix of word-specific mixture weights

\(\alpha\) :

Per-document activity pattern distributions hyperparameter

\(\beta\) :

Per-activity word distribution hyperparameter

MAE:

Mean absolute error

\(v_r\) :

Estimated presence duration for hour r

\({v'}_r\) :

Actual presence duration for hour r

\(\hat{H}\) :

Number of hours

RAE:

Relative absolute error

\(\xi\) :

Training corpus model

\(G_m\) :

Length of the document m

References

  • Aztiria A (2010) Learning frequent behaviours of the users in intelligent environments. J Ambient Intell Smart Environ 2(4):435–436. doi:10.3233/AIS-2010-0084

    Google Scholar 

  • Baggenstoss PM (2001) A modified baum-welch algorithm for hidden markov models with multiple observation spaces. IEEE Trans Speech Audio Process 9(4):411–416. doi:10.1109/89.917686

    Article  Google Scholar 

  • Bickford M (2005) Stress in the workplace: a general overview of the causes, the effects, and the solutions. Canadian Mental Health Association Newfoundland and Labrador Division, pp 1–3

  • Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022

    MATH  Google Scholar 

  • Bo NB, Deboeverie F, Eldib M, Guan J, Xie X, Niño J, Van Haerenborgh D, Slembrouck M, Van de Velde S, Steendam H et al (2014) Human mobility monitoring in very low resolution visual sensor network. Sensors 14(11):20,800–20,824. doi:10.3390/s141120800

  • Camilli M, Kleihorst R (2011) Demo: Mouse sensor networks, the smart camera. In: Fifth ACM/IEEE International Conference on Distributed Smart Cameras, pp 1–3. doi:10.1109/ICDSC.2011.6042944

  • Castanedo F, de Ipia DL, Aghajan HK, Kleihorst R (2014) Learning routines over long-term sensor data using topic models. Expert Syst 31(4):365–377. doi:10.1111/exsy.12033

    Article  Google Scholar 

  • Chen CW, Aghajan H (2011) Multiview social behavior analysis in work environments. In: Fifth ACM/IEEE International Conference on Distributed Smart Cameras, pp 1–6. doi:10.1109/ICDSC.2011.6042910

  • Chen CW, Aztiria A, Aghajan H (2011a) Learning human behaviour patterns in work environments. In: CVPR 2011 WORKSHOPS, pp 47–52. doi:10.1109/CVPRW.2011.5981696

  • Chen CW, Aztiria A, Allouch SB, Aghajan H (2011b) Understanding the influence of social interactions on individuals behavior pattern in a work environment. In: International Workshop on Human Behavior Understanding, Springer, pp 146–157. doi:10.1007/978-3-642-25446-8_16

  • Chen CW, Ugarte RC, Wu C, Aghajan H (2011) Discovering social interactions in real work environments. Face Gesture 2011:933–938. doi:10.1109/FG.2011.5771376

    Google Scholar 

  • Cheng CC, Lee D (2014) Smart sensors enable smart air conditioning control. Sensors 14(6):11,179–11,203. doi:10.3390/s140611179

  • Cinaz B, Arnrich B, Marca R, Tröster G (2013) Monitoring of mental workload levels during an everyday life office-work scenario. Person Ubiquitous Comput 17(2):229–239. doi:10.1007/s00779-011-0466-1

    Article  Google Scholar 

  • Cosemans B, Cosmar M, Gründler R, Flemming D, Van den Broek K (2014) Calculating the cost of work-related stress and psychosocial risks. In: Tech. rep., European Agency for Safety and Health at Work, Luxembourg. doi:10.2802/20493

  • Docobo (2013) Sonopa:social networks for older adults to promote an active life. http://www.sonopa.eu (Online). Accessed 12 May 2016

  • Duong T, Hazelton ML (2005) Cross-validation bandwidth matrices for multivariate kernel density estimation. Scand J Stat 32(3):485–506. doi:10.1111/j.1467-9469.2005.00445.x

    Article  MathSciNet  MATH  Google Scholar 

  • Eijckelhof BH, Huysmans MA, Blatter BM, Leider PC, Johnson PW, van Dien JH, Dennerlein JT, van der Beek AJ (2014) Office workers’ computer use patterns are associated with workplace stressors. Appl Ergon 45(6):1660–1667. doi:10.1016/j.apergo.2014.05.013

    Article  Google Scholar 

  • Eldib M, Bo NB, Deboeverie F, Nino J, Guan J, Van de Velde S, Steendam H, Aghajan H, Philips W (2014a) A low resolution multi-camera system for person tracking. In: IEEE International Conference on Image Processing (ICIP), IEEE, pp 378–382. doi:10.1109/ICIP.2014.7025075

  • Eldib M, Bo NB, Deboeverie F, Xie X, Philips W, Aghajan H (2014b) Behavior analysis for aging-in-place using similarity heatmaps. In: Proceedings of the International Conference on Distributed Smart Cameras, ACM, ICDSC ’14, vol 6, pp 1–34. doi:10.1145/2659021.2659038

  • Eldib M, Deboeverie F, Haerenborgh DV, Philips W, Aghajan H (2015a) Detection of visitors in elderly care using a low-resolution visual sensor network. In: Proceedings of the 9th International Conference on Distributed Smart Cameras, ACM, pp 56–61. doi:10.1145/2789116.2789137

  • Eldib M, Deboeverie F, Philips W, Aghajan H (2015b) Sleep analysis for elderly care using a low-resolution visual sensor network. In: Human Behavior Understanding, Springer, pp 26–38. doi:10.1007/978-3-319-24195-1_3

  • Eldib M, Deboeverie F, Philips W, Aghajan H (2016a) Behavior analysis for elderly care using a network of low-resolution visual sensors. J Electron Imaging 25(4):041,003–041,003. doi:10.1117/1.JEI.25.4.041003

  • Eldib M, Deboeverie F, Philips W, Aghajan H (2016b) Towards more efficient use of office space. In: Proceedings of the 10th International Conference on Distributed Smart Camera, ACM, pp 37–43. doi:10.1145/2967413.2967424

  • Eldib M, Zhang T, Deboeverie F, Philips W, Aghajan H (2016c) A data fusion approach for identifying lifestyle patterns in elderly care. In: Active and Assisted Living: Technologies and Applications, Healthcare Technologies, Institution of Engineering and Technology, pp 81–102. doi:10.1049/PBHE006E_ch5

  • EU-OSHA (2013a) Campaign guide managing stress and psychosocial risks at work. https://www.healthy-workplaces.eu/en/campaign-materials/guide (Online). Accessed 12 May 2016

  • EU-OSHA (2013b) European opinion poll on occupational safety and health. In: Tech. rep. doi:10.2802/55505

  • Farrahi K, Gatica-Perez D (2011) Discovering routines from large-scale human locations using probabilistic topic models. ACM Trans Intell Syst Technol 2(1):3:1–3:27. doi:10.1145/1889681.1889684

  • Griffiths TL, Steyvers M (2004) Finding scientific topics. Proc Natl Acad Sci 101(suppl 1):5228–5235. doi:10.1073/pnas.0307752101

    Article  Google Scholar 

  • Hamid R, Maddi S, Johnson A, Bobick A, Essa I, Isbell C (2009) A novel sequence representation for unsupervised analysis of human activities. Artif Intell 173(14):1221–1244. doi:10.1016/j.artint.2009.05.002

    Article  MathSciNet  Google Scholar 

  • Healey JA, Picard RW (2005) Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans Intell Transport Syst 6(2):156–166. doi:10.1109/TITS.2005.848368

    Article  Google Scholar 

  • Huynh T, Fritz M, Schiele B (2008) Discovery of activity patterns using topic models. In: Proceedings of the 10th International Conference on Ubiquitous Computing, ACM, UbiComp ’08, pp 10–19. doi:10.1145/1409635.1409638

  • iMinds (2013) Little sister: low-cost monitoring for care and retail. https://www.iminds.be/en/projects/littlesister (Online). Accessed 12 May 2016

  • Jaramillo P, Amft O (2013) Improving energy efficiency through activity-aware control of office appliances using proximity sensing-a real-life study. In: IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops), IEEE, pp 664–669. doi:10.1109/PerComW.2013.6529576

  • Kim E, Helal S, Cook D (2010) Human activity recognition and pattern discovery. IEEE Pervasive Comput 9(1):48–53. doi:10.1109/MPRV.2010.7

    Article  Google Scholar 

  • Lafferty J, McCallum A, Pereira F et al (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the eighteenth international conference on machine learning, ICML. Morgan Kaufmann Publishers Inc., vol 1, pp 282–289

  • Lancaster HO, Seneta E (1969) Chi-square distribution. Wiley Online Library. doi:10.1002/0470011815.b2a15018

  • Liao W, Zhang W, Zhu Z, Ji Q (2005) A real-time human stress monitoring system using dynamic bayesian network. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)—Workshops, pp 70–70. doi:10.1109/CVPR.2005.394

  • Liu DC, Nocedal J (1989) On the limited memory bfgs method for large scale optimization. Math Prog 45(1):503–528. doi:10.1007/BF01589116

    Article  MathSciNet  MATH  Google Scholar 

  • Milczarek M, Rial-González E, Schneider E (2009) OSH [Occupational safety and health] in figures: stress at work-facts and figures. Office for Official Publications of the European Communities

  • Milenkovic M, Amft O (2013) An opportunistic activity-sensing approach to save energy in office buildings. In: Proceedings of the fourth international conference on Future energy systems, ACM, pp 247–258. doi:10.1145/2487166.2487194

  • Mrazovac B, Bjelica MZ, Teslic N, Papp I (2011) Towards ubiquitous smart outlets for safety and energetic efficiency of home electric appliances. In: 2011 IEEE International Conference on Consumer Electronics—Berlin (ICCE-Berlin), pp 322–326. doi:10.1109/ICCE-Berlin.2011.6031795

  • Okada Y, Yoto TY, Suzuki T, Sakuragawa S, Sugiura T (2013) Wearable ecg recorder with acceleration sensors for monitoring daily stress: Office work simulation study. In: 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp 4718–4721. doi:10.1109/EMBC.2013.6610601

  • Oliver N, Horvitz E (2005) A comparison of hmms and dynamic bayesian networks for recognizing office activities. In: Proceedings of the 10th International Conference on User Modeling, UM’05, pp 199–209. doi:10.1007/11527886_26

  • Oliver N, Horvitz E, Garg A (2002) Layered representations for human activity recognition. In: Multimodal Interfaces, 2002. Proceedings. Fourth IEEE International Conference on, IEEE, pp 3–8. doi:10.1109/ICMI.2002.1166960

  • Oliver N, Garg A, Horvitz E (2004) Layered representations for learning and inferring office activity from multiple sensory channels. Comput Vis Image Underst 96(2):163–180. doi:10.1016/j.cviu.2004.02.004

    Article  Google Scholar 

  • Rabiner L (1989) A tutorial on hidden markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286. doi:10.1109/5.18626

    Article  Google Scholar 

  • Rish I (2001) An empirical study of the naive bayes classifier. In: IJCAI 2001 workshop on empirical methods in artificial intelligence. IBM New York, vol 3, pp 41–46

  • Salamone F, Belussi L, Danza L, Ghellere M, Meroni I (2016) An open source smart lamp for the optimization of plant systems and thermal comfort of offices. Sensors 16(3):338. doi:10.3390/s16030338

    Article  Google Scholar 

  • Sheather SJ, Jones MC (1991) A reliable data-based bandwidth selection method for kernel density estimation. J R Stat Soc Ser B (Methodol). doi:10.2307/2345597

  • Si Z, Pei M, Yao B, Zhu SC (2011) Unsupervised learning of event and-or grammar and semantics from video. In: International Conference on Computer Vision, IEEE, pp 41–48. doi:10.1109/ICCV.2011.6126223

  • Silverman BW (1986) Density estimation for statistics and data analysis, vol 26. CRC Press. doi:10.1007/978-1-4899-3324-9

  • Simonoff J (1996) Smoothing methods in statistics. Springer, Berlin

  • Sutton C, McCallum A (2012) An introduction to conditional random fields. Found Trends Mach Learn 4(4):267373. doi:10.1561/2200000013

    Article  Google Scholar 

  • Tao S, Kudo M, Nonaka H, Toyama J (2011) Person authentication and activities analysis in an office environment using a sensor network. In: International Joint Conference on Ambient Intelligence, Springer, pp 119–127. doi:10.1007/978-3-642-31479-7_19

  • Teixeira T, Dublon G, Savvides A (2010) A survey of human-sensing: methods for detecting presence, count, location, track, and identity. ENALAB technical report

  • Varadarajan J, Emonet R, Odobez JM (2013) A sequential topic model for mining recurrent activities from long term video logs. Int J Comp Vis 103(1):100–126. doi:10.1007/s11263-012-0596-6

    Article  MathSciNet  MATH  Google Scholar 

  • Wojek C, Nickel K, Stiefelhagen R (2006) Activity recognition and room-level tracking in an office environment. In: IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp 25–30. doi:10.1109/MFI.2006.265608

  • Xie X, Deboeverie F, Eldib M, Philips W, Aghajan H (2014) Phd forum: Analyzing behaviors patterns of the elderly from low-precision trajectories. In: Proceedings of the International Conference on Distributed Smart Cameras, ACM, ICDSC ’14, vol 2, pp 1–47. doi:10.1145/2659021.2675057

  • Ziefle M, Rocker C, Holzinger A (2011) Medical technology in smart homes: exploring the user’s perspective on privacy, intimacy and trust. In: IEEE 35th Annual Computer Software and Applications Conference Workshops, pp 410–415. doi:10.1109/COMPSACW.2011.75

  • Zimmermann P, Guttormsen S, Danuser B, Gomez P (2003) Affective computinga rationale for measuring mood with mouse and keyboard. Int J Occup Saf Ergon 9(4):539–551. doi:10.1080/10803548.2003.11076589

    Article  Google Scholar 

Download references

Acknowledgements

This research has been financed by the Belgian National Fund for Scientific Research (FWO Flanders) and Ghent University through the FWO project G.0.398.11.N.10. The evaluation was performed in the context of the projects “LittleSister” and the European AAL project “SONOPA,” financed by the agency for Innovation by Science and Technology (IWT), imec and the EU Ambient Assisted Living programme.

Author information

Authors and Affiliations

  1. Image Processing and Interpretation, TELIN, Ghent University/imec, Sint Pietersnieuwstraat 41, 9000, Gent, Belgium

    Mohamed Eldib, Francis Deboeverie, Wilfried Philips & Hamid Aghajan

  2. Ambient Intelligence Research Lab, David Packard Building, Stanford, CA, 94305-9515, USA

    Hamid Aghajan

Authors
  1. Mohamed Eldib
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francis Deboeverie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wilfried Philips
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hamid Aghajan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamed Eldib.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Eldib, M., Deboeverie, F., Philips, W. et al. Discovering activity patterns in office environment using a network of low-resolution visual sensors. J Ambient Intell Human Comput 9, 381–411 (2018). https://doi.org/10.1007/s12652-017-0511-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-017-0511-7

Keywords

Search

Navigation