{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,6,13]],"date-time":"2024-06-13T19:49:42Z","timestamp":1718308182692},"reference-count":22,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2018,8,2]],"date-time":"2018-08-02T00:00:00Z","timestamp":1533168000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["16H02841, 16K00259"],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JSAN"],"abstract":"The development of information technology has added many conveniences to our lives. On the other hand, however, we have to deal with various kinds of information, which can be a difficult task for elderly people or those who are not familiar with information devices. A technology to recognize each person\u2019s activity and providing appropriate support based on that activity could be useful for such people. In this paper, we propose a novel fine-grained activity recognition method for user support systems that focuses on identifying the text at which a user is gazing, based on the idea that the content of the text is related to the activity of the user. It is necessary to keep in mind that the meaning of the text depends on its location. To tackle this problem, we propose the simultaneous use of a wearable device and fixed camera. To obtain the global location of the text, we perform image matching using the local features of the images obtained by these two devices. Then, we generate a feature vector based on this information and the content of the text. To show the effectiveness of the proposed approach, we performed activity recognition experiments with six subjects in a laboratory environment.<\/jats:p>","DOI":"10.3390\/jsan7030031","type":"journal-article","created":{"date-parts":[[2018,8,3]],"date-time":"2018-08-03T07:03:15Z","timestamp":1533279795000},"page":"31","source":"Crossref","is-referenced-by-count":2,"title":["Activity Recognition Using Gazed Text and Viewpoint Information for User Support Systems"],"prefix":"10.3390","volume":"7","author":[{"given":"Shun","family":"Chiba","sequence":"first","affiliation":[{"name":"Graduate School of Engineering, Tohoku University, Aoba 6-6-05, Aramaki, Aoba-ku, Sendai 980-8579, Japan"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-5205-0542","authenticated-orcid":false,"given":"Tomo","family":"Miyazaki","sequence":"additional","affiliation":[{"name":"Graduate School of Engineering, Tohoku University, Aoba 6-6-05, Aramaki, Aoba-ku, Sendai 980-8579, Japan"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-3704-4309","authenticated-orcid":false,"given":"Yoshihiro","family":"Sugaya","sequence":"additional","affiliation":[{"name":"Graduate School of Engineering, Tohoku University, Aoba 6-6-05, Aramaki, Aoba-ku, Sendai 980-8579, Japan"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-7706-9995","authenticated-orcid":false,"given":"Shinichiro","family":"Omachi","sequence":"additional","affiliation":[{"name":"Graduate School of Engineering, Tohoku University, Aoba 6-6-05, Aramaki, Aoba-ku, Sendai 980-8579, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2018,8,2]]},"reference":[{"key":"ref_1","unstructured":"Polana, R., and Nelson, R. (1994, January 9\u201313). Recognizing activities. Proceedings of the 12th International Conference on Pattern Recognition, Jerusalem, Israel."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"160","DOI":"10.2197\/ipsjtcva.3.160","article-title":"A single framework for action recognition based on boosted randomized trees","volume":"3","author":"Yamashita","year":"2011","journal-title":"IPSJ Trans. Comput. Vision Appl."},{"key":"ref_3","unstructured":"Chen, O.T.-C., Tsai, C.-H., Manh, H.H., and Lai, W.-C. (September, January 29). Activity recognition using a panoramic camera for homecare. Proceedings of the 14th IEEE International Conference on Advanced Video and Signal Based Surveillance, Lecce, Italy."},{"key":"ref_4","unstructured":"G\u00e1rate, C., Zaidenberg, S., Badie, J., and Br\u00e9mond, F. (2014, January 5\u20138). Group tracking and behavior recognition in long video surveillance sequences. Proceedings of the 2014 International Conference on Computer Vision Theory and Applications, Lisbon, Portugal."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ouchi, K., and Doi, M. (2013, January 8\u201312). Smartphone-based monitoring system for activities of daily living for elderly people and their relatives etc. Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, Zurich, Switzerland.","DOI":"10.1145\/2494091.2494120"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zeng, M., Nguyen, L.T., Yu, B., Mengshoel, O.J., Zhu, J., Wu, P., and Zhang, J. (2014, January 6\u20137). Convolutional neural networks for human activity recognition using mobile sensors. Proceedings of the 2014 6th International Conference on Mobile Computing, Applications and Services, Austin, TX, USA.","DOI":"10.4108\/icst.mobicase.2014.257786"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Pham, C. (2015, January 8\u201310). MobiRAR: real-time human activity recognition using mobile devices. Proceedings of the 2015 Seventh International Conference on Knowledge and Systems Engineering, Ho Chi Minh City, Vietnam.","DOI":"10.1109\/KSE.2015.43"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Xu, H., Liu, J., Hu, H., and Zhang, Y. (2016). Wearable sensor-based human activity recognition method with multi-features extracted from Hilbert-Huang transform. Sensors, 16.","DOI":"10.3390\/s16122048"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Liu, K.-C., Yen, C.-Y., Chang, L.-H., Hsieh, C.-Y., and Chan, C.-T. (2017, January 9\u201312). Wearable sensor-based activity recognition for housekeeping task. Proceedings of the 2017 IEEE 14th International Conference on Wearable and Implantable Body Sensor Networks, Eindhoven, Netherlands.","DOI":"10.1109\/BSN.2017.7936009"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"5315","DOI":"10.1109\/JSEN.2017.2720725","article-title":"An adaptive algorithm to improve energy efficiency in wearable activity recognition systems","volume":"17","author":"Rezaie","year":"2017","journal-title":"IEEE Sens. J."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Twomey, N., Diethe, T., Fafoutis, X., Elsts, A., McConville, R., Flach, P., and Craddock, I. (2018). A comprehensive study of activity recognition using accelerometers. Informatics, 5.","DOI":"10.20944\/preprints201803.0147.v1"},{"key":"ref_12","unstructured":"Wang, Y., Cang, C., and Yu, H. (2016, January 15\u201317). A review of sensor selection, sensor devices and sensor deployment for wearable sensor-based human activity recognition systems. Proceedings of the 10th International Conference on Software, Knowledge, Information Management & Applications, Chengdu, China."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2442","DOI":"10.1109\/JPROC.2012.2200554","article-title":"First-person vision","volume":"100","author":"Kanade","year":"2012","journal-title":"Proc. IEEE."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2984","DOI":"10.1109\/TIP.2015.2438540","article-title":"Egocentric daily activity recognition via multitask clustering","volume":"24","author":"Yan","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Abebe, G., and Cavallaro, A. (2017, January 22\u201329). A long short-term memory convolutional neural network for first-person vision activity recognition. Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops, Venice, Italy.","DOI":"10.1109\/ICCVW.2017.159"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1049\/iet-cvi.2017.0141","article-title":"Using context from inside-out vision for improved activity recognition","volume":"12","author":"Noor","year":"2018","journal-title":"IET Comput. Vision"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Nguyen, T.-H.-C., Nebel, J.-C., and Florez-Revuelta, F. (2016). Recognition of activities of daily living with egocentric vision: A review. Sensors, 16.","DOI":"10.3390\/s16010072"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Smith, R. (2007, January 23\u201326). An overview of the Tesseract OCR engine. Proceedings of the Ninth International Conference on Document Analysis and Recognition, Parana, Brazil.","DOI":"10.1109\/ICDAR.2007.4376991"},{"key":"ref_19","unstructured":"Okazaki, N., and Tsujii, J. (2010, January 23\u201327). Simple and efficient algorithm for approximate dictionary matching. Proceedings of the 23rd International Conference on Computational Linguistics, Beijing, China."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cviu.2007.09.014","article-title":"Speeded-up robust features (SURF)","volume":"110","author":"Bay","year":"2008","journal-title":"Comput. Vision Image Underst."},{"key":"ref_21","first-page":"680","article-title":"High quality image correction algorithm with cubic interpolation and its implementations of dedicated hardware engine for fish-eye lens","volume":"36","author":"Mori","year":"2007","journal-title":"J. Inst. Image Electron. Eng. Jpn."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."}],"container-title":["Journal of Sensor and Actuator Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2224-2708\/7\/3\/31\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,12]],"date-time":"2024-06-12T00:25:29Z","timestamp":1718151929000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2224-2708\/7\/3\/31"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8,2]]},"references-count":22,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2018,9]]}},"alternative-id":["jsan7030031"],"URL":"https:\/\/doi.org\/10.3390\/jsan7030031","relation":{},"ISSN":["2224-2708"],"issn-type":[{"value":"2224-2708","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,8,2]]}}}