{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T16:40:43Z","timestamp":1726850443368},"reference-count":18,"publisher":"Fuji Technology Press Ltd.","issue":"2","funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["21K12704"],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JACIII","J. Adv. Comput. Intell. Intell. Inform."],"published-print":{"date-parts":[[2022,3,20]]},"abstract":"Premature babies are admitted to the neonatal intensive care unit (NICU) for several weeks and are generally placed under high medical supervision. The NICU environment is considered to have a bad influence on the formation of the sleep-wake cycle of the neonate, known as the circadian rhythm, because patient monitoring and treatment equipment emit light and noise throughout the day. In order to improve the neonatal environment, researchers have investigated the effect of light and noise on neonates. There are some methods and devices to measure neonatal alertness, but they place on additional burden on neonatal patients or nurses. Therefore, this study proposes an automatic non-contact neonatal alertness state classification method using video images. The proposed method consists of a face region of interest (ROI) location normalization method, histogram of oriented gradients (HOG) and gradient feature-based feature extraction methods, and a neonatal alertness state classification method using machine learning. Comparison experiments using 14 video images of 7 neonatal subjects showed that the weighted support vector machine (w-SVM) using the HOG feature and averaging merge achieved the highest classification performance (micro-F1 of 0.732). In clinical situations, body movement is evaluated primarily to classify waking states. The additional 4 class classification experiments are conducted by combining waking states into a single class, with results that suggest that the proposed facial expression based classification is suitable for the detailed classification of sleeping states.<\/jats:p>","DOI":"10.20965\/jaciii.2022.p0188","type":"journal-article","created":{"date-parts":[[2022,3,19]],"date-time":"2022-03-19T15:02:06Z","timestamp":1647702126000},"page":"188-195","source":"Crossref","is-referenced-by-count":0,"title":["Automatic Neonatal Alertness State Classification Based on Facial Expression Recognition"],"prefix":"10.20965","volume":"26","author":[{"given":"Kento","family":"Morita","sequence":"first","affiliation":[]},{"given":"Nobu C.","family":"Shirai","sequence":"additional","affiliation":[]},{"given":"Harumi","family":"Shinkoda","sequence":"additional","affiliation":[]},{"given":"Asami","family":"Matsumoto","sequence":"additional","affiliation":[]},{"given":"Yukari","family":"Noguchi","sequence":"additional","affiliation":[]},{"given":"Masako","family":"Shiramizu","sequence":"additional","affiliation":[]},{"given":"Tetsushi","family":"Wakabayashi","sequence":"additional","affiliation":[]},{"name":"Graduate School of Engineering, Mie University 1577 Kurimamachiya-cho, Tsu, Mie 514-8507, Japan","sequence":"additional","affiliation":[]},{"name":"Center for Information Technologies and Networks, Mie University 1577 Kurimamachiya-cho, Tsu, Mie 514-8507, Japan","sequence":"additional","affiliation":[]},{"name":"Kagoshima Immaculate Heart University 2365 Amatatsu-cho, Satsumasendai, Kagoshima 895-0011, Japan","sequence":"additional","affiliation":[]},{"name":"Suzuka University of Medical Science 3500-3 Minamitamagaki, Suzuka, Mie 513-8670, Japan","sequence":"additional","affiliation":[]},{"name":"St. Mary College 422 Tubuku-Honmachi, Kurume, Fukuoka 830-8558, Japan","sequence":"additional","affiliation":[]},{"name":"Kyushu University Hospital 3-5-25 Maidashi, Higashi-ku, Fukuoka, Fukuoka 812-8582, Japan","sequence":"additional","affiliation":[]}],"member":"8550","published-online":{"date-parts":[[2022,3,20]]},"reference":[{"key":"key-10.20965\/jaciii.2022.p0188-1","doi-asserted-by":"crossref","unstructured":"H. Blencowe, S. Cousens, M. Z. Oestergaard, D. Chou, A. B. Moller, R. Narwal, A. Adler, C. V. Garcia, S. Rohde, L. Say, and J. E. Lawn, \u201cNational, regional, and worldwide estimates of preterm birth rates in the year 2010 with time trends since 1990 for selected countries: a systematic analysis and implications,\u201d Lancet, Vol.379, No.9832, pp. 2162-2172, 2012.","DOI":"10.1016\/S0140-6736(12)60820-4"},{"key":"key-10.20965\/jaciii.2022.p0188-2","unstructured":"Ministry of Health, Labour and Welfare, \u201cVital Statistics of Japan \u2013The latest trends\u2013,\u201d 2017."},{"key":"key-10.20965\/jaciii.2022.p0188-3","doi-asserted-by":"crossref","unstructured":"S. Blackburn, \u201cEnvironmental impact of the NICU on developmental outcomes,\u201d J. of Pediatric Nursing, Vol.13, No.5, pp. 279-289, 1998.","DOI":"10.1016\/S0882-5963(98)80013-4"},{"key":"key-10.20965\/jaciii.2022.p0188-4","unstructured":"H. Shinkoda, Y. Kinoshita, R. Mitsutake, F. Ueno, H. Arata, C. Kiyohara, Y. Suetsugu, Y. Koga, K. Anai, M. Shiramizu, M. Ochiai, and T. Kaku, \u201cThe influence of premature infants\/sleep and physiological response under NICU environment (illuminance, noise) \u2013 Seen from circadian variation and comparison of day and night \u2013,\u201d Mie Nursing J., Vol.17, No.1, pp. 35-44, 2015 (in Japanese)."},{"key":"key-10.20965\/jaciii.2022.p0188-5","unstructured":"M. Shiramizu and H. Shinkoda, \u201cA pilot study to examine the most suitable lighting environment for the premature infants: Analysis of amount of activity and physiological response by using Actigraph,\u201d Mie Nursing J., Vol.18, No.1, pp. 15-21, 2016 (in Japanese)."},{"key":"key-10.20965\/jaciii.2022.p0188-6","unstructured":"T. B. Brazelton, \u201cNeonatal Behavioral Assessment Scale,\u201d Spastics International Medical Publications, 1973."},{"key":"key-10.20965\/jaciii.2022.p0188-7","doi-asserted-by":"crossref","unstructured":"V. Ib\u00e1\u00f1ez, J. Silva, and O. Cauli, \u201cA survey on sleep assessment methods,\u201d PeerJ, Vol.6, Article No.e4849, 2018.","DOI":"10.7717\/peerj.4849"},{"key":"key-10.20965\/jaciii.2022.p0188-8","doi-asserted-by":"crossref","unstructured":"W. C. Orr, \u201cUtilization of Polysomnography in the Assessment of Sleep Disorders,\u201d Medical Clinics of North America, Vol.69, No.6, pp. 1153-1167, 1985.","DOI":"10.1016\/S0025-7125(16)30980-4"},{"key":"key-10.20965\/jaciii.2022.p0188-9","doi-asserted-by":"crossref","unstructured":"R. J. Cole, D. F. Kripke, W. Gruen, D. J. Mullaney, and J. C. Gillin, \u201cAutomatic Sleep\/Wake Identification From Wrist Activity,\u201d Sleep, Vol.15, Issue 5, pp. 461-469, 1992.","DOI":"10.1093\/sleep\/15.5.461"},{"key":"key-10.20965\/jaciii.2022.p0188-10","doi-asserted-by":"crossref","unstructured":"M. Sung, T. M. Adamson, and R. S. C. Horne, \u201cValidation of actigraphy for determining sleep and wake in preterm infants,\u201d Acta Paediatrica, Vol.98, Issue 1, pp. 52-57, 2009.","DOI":"10.1111\/j.1651-2227.2008.01002.x"},{"key":"key-10.20965\/jaciii.2022.p0188-11","unstructured":"E. Nakayama, H. Kobayashi, and N. Yamamoto, \u201cA Basic Study on Sleep-Wake Identification by Wrist Actigraph,\u201d Ishikawa J. of Nursing, Vol.3, No.2, pp. 31-37, 2006 (in Japanese)."},{"key":"key-10.20965\/jaciii.2022.p0188-12","doi-asserted-by":"crossref","unstructured":"S. Cabon, F. Por\u00e9e, A. Simon, B. Met-Montot, P. Pladys, O. Rosec, N. Nardi, and G. Carrault, \u201cAudio- and video-based estimation of the sleep stages of newborns in Neonatal Intensive Care Unit,\u201d Biomedical Signal Processing and Control, Vol.52, pp. 362-370, 2019.","DOI":"10.1016\/j.bspc.2019.04.011"},{"key":"key-10.20965\/jaciii.2022.p0188-13","doi-asserted-by":"crossref","unstructured":"H. F. R. Prechtl, \u201cThe behavioural states of the newborn infant (a review),\u201d Brain Research, Vol.76, No.2, pp. 185-212, 1974.","DOI":"10.1016\/0006-8993(74)90454-5"},{"key":"key-10.20965\/jaciii.2022.p0188-14","unstructured":"M. Hattori, K. Morita, T. Wakabayashi, H. Shinkoda, A. Matsumoto, Y. Noguchi, and M. Shiramizu, \u201cNeonatal Sleep-Wake States Estimation in the NICU by Body Movement Detection,\u201d Int. Workshop on Frontiers of Computer Vision (IW-FCV), 2021."},{"key":"key-10.20965\/jaciii.2022.p0188-15","unstructured":"F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and \u00c9. Duchesnay, \u201cScikit-learn: Machine Learning in Python,\u201d J. of Machine Learning Research, Vol.12, pp. 2825-2830, 2011."},{"key":"key-10.20965\/jaciii.2022.p0188-16","doi-asserted-by":"crossref","unstructured":"N. Dalal and T. Bill, \u201cHistograms of oriented gradients for human detection,\u201d 2005 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR\u201905), Vol.1, 2005.","DOI":"10.1109\/CVPR.2005.177"},{"key":"key-10.20965\/jaciii.2022.p0188-17","doi-asserted-by":"crossref","unstructured":"T. Wakabayashi, S. Tsuruoka, F. Kimura, and Y. Miyake, \u201cIncreasing the feature size in handwritten numeral recognition to improve accuracy,\u201d Systems and Computers in Japan, Vol.26, No.8, pp. 35-44, 1995.","DOI":"10.1002\/scj.4690260804"},{"key":"key-10.20965\/jaciii.2022.p0188-18","doi-asserted-by":"crossref","unstructured":"M. Shi, Y. Fujisawa, T. Wakabayashi, and F. Kimura, \u201cHandwritten numeral recognition using gradient and curvature of gray scale image,\u201d Pattern Recognition, Vol.35, No.10, pp. 2051-2059, 2002.","DOI":"10.1016\/S0031-3203(01)00203-5"}],"container-title":["Journal of Advanced Computational Intelligence and Intelligent Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.fujipress.jp\/main\/wp-content\/themes\/Fujipress\/hyosetsu.php?ppno=JACII002600020008","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T16:20:36Z","timestamp":1726849236000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.fujipress.jp\/jaciii\/jc\/jacii002600020188"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,20]]},"references-count":18,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,3,20]]},"published-print":{"date-parts":[[2022,3,20]]}},"URL":"https:\/\/doi.org\/10.20965\/jaciii.2022.p0188","relation":{},"ISSN":["1883-8014","1343-0130"],"issn-type":[{"type":"electronic","value":"1883-8014"},{"type":"print","value":"1343-0130"}],"subject":[],"published":{"date-parts":[[2022,3,20]]}}}