{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T13:14:21Z","timestamp":1722950061405},"reference-count":56,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2017,12,20]],"date-time":"2017-12-20T00:00:00Z","timestamp":1513728000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"crossref","award":["61432019 and 61632019"],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2018,2,28]]},"abstract":"In sign language recognition (SLR) with multimodal data, a sign word can be represented by multiply features, for which there exist an intrinsic property and a mutually complementary relationship among them. To fully explore those relationships, we propose an online early-late fusion method based on the adaptive Hidden Markov Model (HMM). In terms of the intrinsic property, we discover that inherent latent change states of each sign are related not only to the number of key gestures and body poses but also to their translation relationships. We propose an adaptive HMM method to obtain the hidden state number of each sign by affinity propagation clustering. For the complementary relationship, we propose an online early-late fusion scheme. The early fusion (feature fusion) is dedicated to preserving useful information to achieve a better complementary score, while the late fusion (score fusion) uncovers the significance of those features and aggregates them in a weighting manner. Different from classical fusion methods, the fusion is query adaptive. For different queries, after feature selection (including the combined feature), the fusion weight is inversely proportional to the area under the curve of the normalized query score list for each selected feature. The whole fusion process is effective and efficient. Experiments verify the effectiveness on the signer-independent SLR with large vocabulary. Compared either on different dataset sizes or to different SLR models, our method demonstrates consistent and promising performance.<\/jats:p>","DOI":"10.1145\/3152121","type":"journal-article","created":{"date-parts":[[2017,12,20]],"date-time":"2017-12-20T14:54:00Z","timestamp":1513781640000},"page":"1-18","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":32,"title":["Online Early-Late Fusion Based on Adaptive HMM for Sign Language Recognition"],"prefix":"10.1145","volume":"14","author":[{"ORCID":"http:\/\/orcid.org\/0000-0003-2594-254X","authenticated-orcid":false,"given":"Dan","family":"Guo","sequence":"first","affiliation":[{"name":"Hefei University of Technology, Hefei, P. R. China"}]},{"given":"Wengang","family":"Zhou","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, P. R. China"}]},{"given":"Houqiang","family":"Li","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, P. R. China"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-3094-7735","authenticated-orcid":false,"given":"Meng","family":"Wang","sequence":"additional","affiliation":[{"name":"Hefei University of Technology, Hefei, P. R. China"}]}],"member":"320","published-online":{"date-parts":[[2017,12,20]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2008.203"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/938978.939161"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2015.2505089"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR.2016.7899606"},{"key":"e_1_2_1_5_1","volume-title":"International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. 620--625","author":"Celebi Sait","year":"2013"},{"key":"e_1_2_1_6_1","volume-title":"Return of the devil in the details: Delving deep into convolutional nets. Arxiv Preprint Arxiv:1405.3531","author":"Chatfield Ken","year":"2014"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2015.2469551"},{"key":"e_1_2_1_8_1","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition Workshops. 44--52","author":"Dong Cao","year":"2015"},{"key":"e_1_2_1_9_1","volume-title":"European Conference on Computer Vision Workshop. 459--473","author":"Escalera Sergio","year":"2014"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.213"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208303"},{"key":"e_1_2_1_12_1","volume-title":"Frey and Delbert Dueck","author":"Brendan","year":"2007"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2016.7532885"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2015.7177428"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2005.01.012"},{"key":"e_1_2_1_16_1","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition. 3306--3313","author":"Khan Fahad Shahbaz"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-011-0495-2"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.667881"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.98"},{"key":"e_1_2_1_20_1","volume-title":"International Conference on Neural Information Processing Systems. 1097--1105","author":"Krizhevsky Alex"},{"key":"e_1_2_1_21_1","volume-title":"European Signal Processing Conference. 1975--1979","author":"Kurakin Alexey","year":"2012"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3089250"},{"key":"e_1_2_1_23_1","volume-title":"Asian Conference on Computer Vision. 233--246","author":"Lin Yushun","year":"2014"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2016.7532884"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995315"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2015.2399851"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.456"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.70796"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2013.69"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2461544"},{"key":"e_1_2_1_31_1","volume-title":"British Machine Vision Conference.","author":"Pfister Tomas","year":"2013"},{"key":"e_1_2_1_32_1","volume-title":"Sander Dieleman, Mieke Van Herreweghe, and Joni Dambre.","author":"Pigou Lionel","year":"2016"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.3390\/s17061341"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2072298.2071946"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.5555\/1367985.1367993"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2013.2265337"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2629481"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818708"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2008.224"},{"key":"e_1_2_1_40_1","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition Workshops. 56--64","author":"Wan Jun"},{"key":"e_1_2_1_41_1","volume-title":"IEEE Conference and Workshops on Automatic Face and Gesture Recognition. 1--6.","author":"Wang Hanjie","year":"2015"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.5555\/2354409.2354966"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.5555\/1641661.1641671"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2012.2207397"},{"key":"e_1_2_1_45_1","first-page":"1","article-title":"First-person daily activity recognition with manipulated object proposals and non-linear feature fusion","volume":"99","author":"Wang Meng","year":"2017","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2537340"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654931"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2964284.2964328"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2962719"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038917"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2016.7552950"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2733373.2806224"},{"key":"e_1_2_1_53_1","volume-title":"Asian Conference on Computer Vision. 65--80","author":"Zhang Qilin","year":"2014"},{"key":"e_1_2_1_54_1","volume-title":"European Conference on Computer Vision. 660--673","author":"Zhang Shaoting"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2648583"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298783"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3152121","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,31]],"date-time":"2022-12-31T19:47:31Z","timestamp":1672516051000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3152121"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,12,20]]},"references-count":56,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2018,2,28]]}},"alternative-id":["10.1145\/3152121"],"URL":"https:\/\/doi.org\/10.1145\/3152121","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,12,20]]},"assertion":[{"value":"2017-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-12-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}