{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,2,13]],"date-time":"2024-02-13T14:28:43Z","timestamp":1707834523334},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2019,10,29]],"date-time":"2019-10-29T00:00:00Z","timestamp":1572307200000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100004344","name":"Adobe Systems","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004344","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1700832"],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2018,11,30]]},"abstract":"Predicting users\u2019 proficiencies is a critical component of AI-powered personal assistants. This article introduces a novel approach for the prediction based on users\u2019 diverse, noisy, and passively generated application usage histories. We propose a novel bi-directional recurrent neural network with hierarchical attention mechanism to extract sequential patterns and distinguish informative traces from noise. Our model is able to attend to the most discriminative actions and sessions to make more accurate and directly interpretable predictions while requiring 50\u00d7 less training data than the state-of-the-art sequential learning approach. We evaluate our model with two large scale datasets collected from 68K Photoshop users: a digital design skill dataset where the user skill is determined by the quality of the end products and a software skill dataset where users self-disclose their software usage skill levels. The empirical results demonstrate our model\u2019s superior performance compared to existing user representation learning techniques that leverage action frequencies and sequential patterns. In addition, we qualitatively illustrate the model\u2019s significant interpretative power. The proposed approach is broadly relevant to applications that generate user time-series analytics.<\/jats:p>","DOI":"10.1145\/3232231","type":"journal-article","created":{"date-parts":[[2018,10,29]],"date-time":"2018-10-29T12:02:18Z","timestamp":1540814538000},"page":"1-18","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Characterizing User Skills from Application Usage Traces with Hierarchical Attention Recurrent Networks"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-6615-8615","authenticated-orcid":false,"given":"Longqi","family":"Yang","sequence":"first","affiliation":[{"name":"Cornell Tech, Cornell University, New York, NY"}]},{"given":"Chen","family":"Fang","sequence":"additional","affiliation":[{"name":"Adobe Research, Park Avenue, San Jose, CA"}]},{"given":"Hailin","family":"Jin","sequence":"additional","affiliation":[{"name":"Adobe Research, Park Avenue, San Jose, CA"}]},{"given":"Matthew D.","family":"Hoffman","sequence":"additional","affiliation":[{"name":"Google, San Francisco, CA"}]},{"given":"Deborah","family":"Estrin","sequence":"additional","affiliation":[{"name":"Cornell Tech, Cornell University, New York, NY"}]}],"member":"320","published-online":{"date-parts":[[2018,10,29]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467.","author":"Abadi Mart\u00edn","year":"2016","unstructured":"Mart\u00edn Abadi , Ashish Agarwal , Paul Barham , Eugene Brevdo , Zhifeng Chen , Craig Citro , Greg S. Corrado , Andy Davis , Jeffrey Dean , Matthieu Devin , 2016 . Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467. Mart\u00edn Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. 2016. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2792838.2799496"},{"key":"e_1_2_1_3_1","unstructured":"Dzmitry Bahdanau Kyunghyun Cho and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473. Dzmitry Bahdanau Kyunghyun Cho and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1961189.1961199"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the 5th Workshop on Human--Computer Interaction and Information Retrieval (HCIR).","author":"Cole Michael J.","year":"2011","unstructured":"Michael J. Cole , Jacek Gwizdka , Nicholas J. Belkin , and Chang Liu . 2011 . User domain knowledge and eye movement patterns during search . In Proceedings of the 5th Workshop on Human--Computer Interaction and Information Retrieval (HCIR). Michael J. Cole, Jacek Gwizdka, Nicholas J. Belkin, and Chang Liu. 2011. User domain knowledge and eye movement patterns during search. In Proceedings of the 5th Workshop on Human--Computer Interaction and Information Retrieval (HCIR)."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2010128"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063639"},{"key":"e_1_2_1_8_1","unstructured":"Cristian Danescu-Niculescu-Mizil Moritz Sudhof Dan Jurafsky Jure Leskovec and Christopher Potts. 2013. A computational approach to politeness with application to social factors. arXiv:1306.6078. Cristian Danescu-Niculescu-Mizil Moritz Sudhof Dan Jurafsky Jure Leskovec and Christopher Potts. 2013. A computational approach to politeness with application to social factors. arXiv:1306.6078."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047220"},{"key":"e_1_2_1_10_1","volume-title":"Greedy function approximation: A gradient boosting machine. Annals of Statistics","author":"Friedman Jerome H.","year":"2001","unstructured":"Jerome H. Friedman . 2001. Greedy function approximation: A gradient boosting machine. Annals of Statistics ( 2001 ), 1189--1232. Jerome H. Friedman. 2001. Greedy function approximation: A gradient boosting machine. Annals of Statistics (2001), 1189--1232."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-010-9073-5"},{"key":"e_1_2_1_12_1","first-page":"111","article-title":"Leveraging educational data mining for real-time performance assessment of scientific inquiry skills within microworlds","volume":"4","author":"Gobert Janice D.","year":"2012","unstructured":"Janice D. Gobert , Michael A. Sao Pedro , Ryan S. J. D. Baker , Ermal Toto , and Orlando Montalvo . 2012 . Leveraging educational data mining for real-time performance assessment of scientific inquiry skills within microworlds . JEDM-Journal of Educational Data Mining 4 , 1 (2012), 111 -- 143 . Janice D. Gobert, Michael A. Sao Pedro, Ryan S. J. D. Baker, Ermal Toto, and Orlando Montalvo. 2012. Leveraging educational data mining for real-time performance assessment of scientific inquiry skills within microworlds. JEDM-Journal of Educational Data Mining 4, 1 (2012), 111--143.","journal-title":"JEDM-Journal of Educational Data Mining"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2013.6638947"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2788627"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3109859.3109882"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 256--265","author":"Horvitz Eric","year":"1998","unstructured":"Eric Horvitz , Jack Breese , David Heckerman , David Hovel , and Koos Rommelse . 1998 . The lumiere project: Bayesian user modeling for inferring the goals and needs of software users . In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 256--265 . Eric Horvitz, Jack Breese, David Heckerman, David Hovel, and Koos Rommelse. 1998. The lumiere project: Bayesian user modeling for inferring the goals and needs of software users. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 256--265."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1111\/tops.12251"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470753"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775067"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1719970.1720023"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2010086"},{"key":"e_1_2_1_23_1","unstructured":"Guolin Ke Qi Meng Thomas Finley Taifeng Wang Wei Chen Weidong Ma Qiwei Ye and Tie-Yan Liu. 2017. LightGBM: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems. 3149--3157. Guolin Ke Qi Meng Thomas Finley Taifeng Wang Wei Chen Weidong Ma Qiwei Ye and Tie-Yan Liu. 2017. LightGBM: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems. 3149--3157."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2678025.2701387"},{"key":"e_1_2_1_25_1","volume-title":"Adam: A method for stochastic optimization. arXiv:1412.6980.","author":"Kingma Diederik","year":"2014","unstructured":"Diederik Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv:1412.6980. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980."},{"key":"e_1_2_1_26_1","volume-title":"Hinton","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E . Hinton . 2012 . Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems . 1097--1105. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097--1105."},{"key":"e_1_2_1_27_1","doi-asserted-by":"crossref","unstructured":"Tao Lei Hrishikesh Joshi Regina Barzilay Tommi Jaakkola Katerina Tymoshenko Alessandro Moschitti and Lluis Marquez. 2015. Semi-supervised question retrieval with gated convolutions. arXiv:1512.05726. Tao Lei Hrishikesh Joshi Regina Barzilay Tommi Jaakkola Katerina Tymoshenko Alessandro Moschitti and Lluis Marquez. 2015. Semi-supervised question retrieval with gated convolutions. arXiv:1512.05726.","DOI":"10.18653\/v1\/N16-1153"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1970378.1970380"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1002\/meet.14504901134"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000016"},{"key":"e_1_2_1_31_1","volume-title":"How does domain expertise affect users","author":"Mao Jiaxin","unstructured":"Jiaxin Mao , Yiqun Liu , Min Zhang , and Shaoping Ma. 2016. How does domain expertise affect users \u2019 search processes in exploratory searches? In SAL@ SIGIR. Jiaxin Mao, Yiqun Liu, Min Zhang, and Shaoping Ma. 2016. How does domain expertise affect users\u2019 search processes in exploratory searches? In SAL@ SIGIR."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2488388.2488466"},{"key":"e_1_2_1_33_1","unstructured":"Tomas Mikolov Ilya Sutskever Kai Chen Greg S. Corrado and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111--3119. Tomas Mikolov Ilya Sutskever Kai Chen Greg S. Corrado and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111--3119."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/238386.238541"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2930238.2930264"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047199"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/78.650093"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2670313"},{"key":"e_1_2_1_39_1","volume-title":"Le","author":"Sutskever Ilya","year":"2014","unstructured":"Ilya Sutskever , Oriol Vinyals , and Quoc V . Le . 2014 . Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems . 3104--3112. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems. 3104--3112."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298935"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1498759.1498819"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3018661.3018689"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3054183"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072614"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-1174"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208312"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2010131"},{"key":"e_1_2_1_48_1","volume-title":"Advances in Information Systems and Technologies","author":"Zhang Xiangmin","unstructured":"Xiangmin Zhang , Jingjing Liu , and Michael Cole . 2013. Task topic knowledge vs. background domain knowledge: Impact of two types of knowledge on user search performance . In Advances in Information Systems and Technologies . Springer , 179--191. Xiangmin Zhang, Jingjing Liu, and Michael Cole. 2013. Task topic knowledge vs. background domain knowledge: Impact of two types of knowledge on user search performance. In Advances in Information Systems and Technologies. Springer, 179--191."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11280-015-0364-y"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3232231","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3232231","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,31]],"date-time":"2022-12-31T20:03:42Z","timestamp":1672517022000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3232231"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,10,29]]},"references-count":49,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2018,11,30]]}},"alternative-id":["10.1145\/3232231"],"URL":"https:\/\/doi.org\/10.1145\/3232231","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,10,29]]},"assertion":[{"value":"2017-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-10-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}