{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,7,30]],"date-time":"2024-07-30T19:13:58Z","timestamp":1722366838811},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,3,19]],"date-time":"2022-03-19T00:00:00Z","timestamp":1647648000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,19]],"date-time":"2022-03-19T00:00:00Z","timestamp":1647648000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004070","name":"Khalifa University of Science, Technology and Research","doi-asserted-by":"publisher","award":["RCII-2019-002"],"id":[{"id":"10.13039\/501100004070","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"published-print":{"date-parts":[[2022,12]]},"abstract":"Abstract<\/jats:title>\n Introduction<\/jats:title>\n Fast-emerging technologies are making the job market dynamic, causing desirable skills to evolve continuously. It is therefore important to understand the transitions in the job market to proactively identify skill sets required.<\/jats:p>\n <\/jats:sec>\n Case description<\/jats:title>\n A novel data-driven approach is developed to identify trending jobs through a case study in the oil and gas industry. The proposed approach leverages a range of data analytics tools, including Latent Semantic Indexing (LSI), Latent Dirichlet Allocation (LDA), Factor Analysis and Non-Negative Matrix Factorization (NMF), to study changes in the market. Further, our approach is capable of identifying disparities between skills that are covered by the educational system, and the skills that are required in the job market.<\/jats:p>\n <\/jats:sec>\n Discussion and evaluation<\/jats:title>\n The results of the case study show that, while the jobs most likely to be replaced are generally low-skilled, some high-skilled jobs may also be at risk. In addition, mismatches are identified between skills that are imparted by the education system and the skills required in the job market.<\/jats:p>\n <\/jats:sec>\n Conclusions<\/jats:title>\n This study presents how job market and skills required evolved over time, which can help decision-makers to prepare the workforce for highly demanding jobs and skills. Our findings are in line with the concerns that automation is decreasing the demand for certain skills. On the other hand, we also identify the new skills that are required to strengthen the need for collaboration between minds and machines.<\/jats:p>\n <\/jats:sec>","DOI":"10.1186\/s40537-022-00576-5","type":"journal-article","created":{"date-parts":[[2022,3,19]],"date-time":"2022-03-19T17:02:36Z","timestamp":1647709356000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Evaluation of the trends in jobs and skill-sets using data analytics: a case study"],"prefix":"10.1186","volume":"9","author":[{"ORCID":"http:\/\/orcid.org\/0000-0001-8747-2593","authenticated-orcid":false,"given":"Armin","family":"Alibasic","sequence":"first","affiliation":[]},{"given":"Himanshu","family":"Upadhyay","sequence":"additional","affiliation":[]},{"given":"Mecit Can Emre","family":"Simsekler","sequence":"additional","affiliation":[]},{"given":"Thomas","family":"Kurfess","sequence":"additional","affiliation":[]},{"given":"Wei Lee","family":"Woon","sequence":"additional","affiliation":[]},{"given":"Mohammed Atif","family":"Omar","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,3,19]]},"reference":[{"issue":"3","key":"576_CR1","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1023\/A:1019049609350","volume":"3","author":"MD Rossetti","year":"2000","unstructured":"Rossetti MD, Felder RA, Kumar A. Simulation of robotic courier deliveries in hospital distribution services. Health Care Manag Sci. 2000;3(3):201\u201313.","journal-title":"Health Care Manag Sci"},{"issue":"2","key":"576_CR2","first-page":"83","volume":"21","author":"W Halal","year":"2017","unstructured":"Halal W, Kolber J, Davies O, Global T. Forecasts of AI and future jobs in 2030: muddling through likely, with two alternative scenarios. J Future Stud. 2017;21(2):83\u201396.","journal-title":"J Future Stud"},{"key":"576_CR3","volume-title":"Mind children: the future of robot and human intelligence","author":"H Moravec","year":"1988","unstructured":"Moravec H. Mind children: the future of robot and human intelligence. Cambridge: Harvard University Press; 1988."},{"key":"576_CR4","volume-title":"Race against the machine: how the digital revolution is accelerating innovation, driving productivity, and irreversibly transforming employment and the economy","author":"E Brynjolfsson","year":"2012","unstructured":"Brynjolfsson E, McAfee A. Race against the machine: how the digital revolution is accelerating innovation, driving productivity, and irreversibly transforming employment and the economy. Brynjolfsson and McAfee; 2012."},{"key":"576_CR5","volume-title":"The second machine age: work, progress, and prosperity in a time of brilliant technologies","author":"E Brynjolfsson","year":"2014","unstructured":"Brynjolfsson E, McAfee A. The second machine age: work, progress, and prosperity in a time of brilliant technologies. New York: WW Norton & Company; 2014."},{"key":"576_CR6","unstructured":"MacCrory F, Westerman G, Alhammadi Y, Brynjolfsson E. Racing with and against the machine: changes in occupational skill composition in an era of rapid technological advance (2014)"},{"key":"576_CR7","doi-asserted-by":"crossref","unstructured":"Ill\u00e9ssy M, Mak\u00f3 C. Automation and creativity in work: Which jobs are at risk of automation? Intersections. East Eur J Soc Polit. 2020;6(2).","DOI":"10.17356\/ieejsp.v6i2.625"},{"issue":"5","key":"576_CR8","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1007\/s00500-002-0187-5","volume":"6","author":"R Kowalczyk","year":"2002","unstructured":"Kowalczyk R. Fuzzy e-negotiation agents. Soft Comput. 2002;6(5):337\u201347.","journal-title":"Soft Comput"},{"issue":"4","key":"576_CR9","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1007\/s12369-009-0030-6","volume":"1","author":"E Broadbent","year":"2009","unstructured":"Broadbent E, Stafford R, MacDonald B. Acceptance of healthcare robots for the older population: review and future directions. Int J Soc Robot. 2009;1(4):319.","journal-title":"Int J Soc Robot"},{"key":"576_CR10","doi-asserted-by":"publisher","first-page":"102002","DOI":"10.1016\/j.labeco.2021.102002","volume":"71","author":"L Alekseeva","year":"2021","unstructured":"Alekseeva L, Azar J, Gine M, Samila S, Taska B. The demand for AI skills in the labor market. Labour Econ. 2021;71:102002.","journal-title":"Labour Econ"},{"issue":"7","key":"576_CR11","doi-asserted-by":"publisher","first-page":"1263","DOI":"10.1007\/s00500-012-0963-9","volume":"17","author":"C-S Lee","year":"2013","unstructured":"Lee C-S, Wang M-H, Wu M-J, Nakagawa Y, Tsuji H, Yamazaki Y, Hirota K. Soft-computing-based emotional expression mechanism for game of computer go. Soft Comput. 2013;17(7):1263\u201382.","journal-title":"Soft Comput"},{"key":"576_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.is.2016.10.009","volume":"65","author":"I Karakatsanis","year":"2017","unstructured":"Karakatsanis I, AlKhader W, MacCrory F, Alibasic A, Omar MA, Aung Z, Woon WL. Data mining approach to monitoring the requirements of the job market: a case study. Inf Syst. 2017;65:1\u20136.","journal-title":"Inf Syst"},{"key":"576_CR13","doi-asserted-by":"crossref","unstructured":"Woon, W.L., Aung, Z., AlKhader, W., Svetinovic, D., Omar, M.A.: Changes in occupational skills-a case study using non-negative matrix factorization. In: International Conference on Neural Information Processing, pp. 627\u2013634. Springer (2015)","DOI":"10.1007\/978-3-319-26555-1_71"},{"key":"576_CR14","unstructured":"Booz M. These 3 industries have the highest talent turnover rates. LinkedIn Talent Blog. Business LinkedIn. 2018."},{"issue":"7","key":"576_CR15","doi-asserted-by":"publisher","first-page":"4959","DOI":"10.1007\/s00500-019-04247-1","volume":"24","author":"A Alibasic","year":"2020","unstructured":"Alibasic A, Simsekler MCE, Kurfess T, Woon WL, Omar MA. Utilizing data science techniques to analyze skill and demand changes in healthcare occupations: case study on USA and UAE healthcare sector. Soft Comput. 2020;24(7):4959\u201376.","journal-title":"Soft Comput"},{"issue":"6","key":"576_CR16","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1038\/nrg3208","volume":"13","author":"PB Jensen","year":"2012","unstructured":"Jensen PB, Jensen LJ, Brunak S. Mining electronic health records: towards better research applications and clinical care. Nat Rev Genet. 2012;13(6):395.","journal-title":"Nat Rev Genet"},{"issue":"9","key":"576_CR17","doi-asserted-by":"publisher","first-page":"3411","DOI":"10.1007\/s00500-015-1812-4","volume":"20","author":"J Li","year":"2016","unstructured":"Li J, Fong S, Zhuang Y, Khoury R. Hierarchical classification in text mining for sentiment analysis of online news. Soft Comput. 2016;20(9):3411\u201320.","journal-title":"Soft Comput"},{"issue":"1","key":"576_CR18","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1007\/s10115-014-0806-3","volume":"46","author":"D Rajpathak","year":"2016","unstructured":"Rajpathak D, De S. A data-and ontology-driven text mining-based construction of reliability model to analyze and predict component failures. Knowl Inf Syst. 2016;46(1):87\u2013113.","journal-title":"Knowl Inf Syst"},{"key":"576_CR19","unstructured":"Br\u00fcning, N., Mangeol, P.: What skills do employers seek in graduates?: Using online job posting data to support policy and practice in higher education (2020)"},{"key":"576_CR20","volume-title":"Advanced data mining techniques","author":"DL Olson","year":"2008","unstructured":"Olson DL, Delen D. Advanced data mining techniques. Berlin: Springer; 2008."},{"key":"576_CR21","volume-title":"Data mining: concepts and techniques","author":"J Han","year":"2011","unstructured":"Han J, Pei J, Kamber M. Data mining: concepts and techniques. Amsterdam: Elsevier; 2011."},{"key":"576_CR22","doi-asserted-by":"crossref","unstructured":"Sarkar D. Processing and understanding text. In: Text Analytics with Python, Springer. 2016, p. 107\u2013165.","DOI":"10.1007\/978-1-4842-2388-8_3"},{"issue":"2","key":"576_CR23","doi-asserted-by":"publisher","first-page":"168","DOI":"10.1017\/pan.2017.44","volume":"26","author":"MJ Denny","year":"2018","unstructured":"Denny MJ, Spirling A. Text preprocessing for unsupervised learning: why it matters, when it misleads, and what to do about it. Polit Anal. 2018;26(2):168\u201389.","journal-title":"Polit Anal"},{"key":"576_CR24","volume-title":"Factor analysis of data matrices","author":"P Horst","year":"1965","unstructured":"Horst P. Factor analysis of data matrices. New York: Holt, Rinehart and Winston; 1965."},{"key":"576_CR25","volume-title":"Factor analysis as a statistical method","author":"DN Lawley","year":"1971","unstructured":"Lawley DN, Maxwell AE. Factor analysis as a statistical method, vol. 18. Hoboken: Wiley Online Library; 1971."},{"key":"576_CR26","doi-asserted-by":"crossref","unstructured":"Jolliffe I. Principal component analysis. Encyclopedia of statistics in behavioral science (2005)","DOI":"10.1002\/0470013192.bsa501"},{"issue":"2","key":"576_CR27","doi-asserted-by":"publisher","first-page":"79","DOI":"10.20982\/tqmp.09.2.p079","volume":"9","author":"AG Yong","year":"2013","unstructured":"Yong AG, Pearce S. A beginner\u2019s guide to factor analysis: focusing on exploratory factor analysis. Tutor Quant Methods Psychol. 2013;9(2):79\u201394.","journal-title":"Tutor Quant Methods Psychol"},{"issue":"6755","key":"576_CR28","doi-asserted-by":"publisher","first-page":"788","DOI":"10.1038\/44565","volume":"401","author":"DD Lee","year":"1999","unstructured":"Lee DD, Seung HS. Learning the parts of objects by non-negative matrix factorization. Nature. 1999;401(6755):788.","journal-title":"Nature"},{"key":"576_CR29","doi-asserted-by":"crossref","unstructured":"Dumais ST, Furnas GW, Landauer TK, Deerwester S, Harshman R: Using latent semantic analysis to improve access to textual information. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 281\u2013285 (1988). ACM","DOI":"10.1145\/57167.57214"},{"issue":"6","key":"576_CR30","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1002\/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9","volume":"41","author":"S Deerwester","year":"1990","unstructured":"Deerwester S, Dumais ST, Furnas GW, Landauer TK, Harshman R. Indexing by latent semantic analysis. J Am Soc Inf Sci. 1990;41(6):391.","journal-title":"J Am Soc Inf Sci"},{"key":"576_CR31","unstructured":"Rehurek R, Sojka P. Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pp. 45\u201350. ELRA, Valletta, Malta. 2010."},{"key":"576_CR32","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511809071","volume-title":"Introduction to information retrieval","author":"CD Manning","year":"2008","unstructured":"Manning CD, Raghavan P, Sch\u00fctze H, et al. Introduction to information retrieval, vol. 1. Cambridge: Cambridge University Press Cambridge; 2008."},{"key":"576_CR33","doi-asserted-by":"crossref","unstructured":"Bradford, R.B.: An empirical study of required dimensionality for large-scale latent semantic indexing applications. In: Proceedings of the 17th ACM Conference on Information and Knowledge Management, pp. 153\u2013162 (2008). ACM.","DOI":"10.1145\/1458082.1458105"},{"key":"576_CR34","first-page":"993","volume":"3","author":"DM Blei","year":"2003","unstructured":"Blei DM, Ng AY, Jordan MI. Latent Dirichlet Allocation. J Mach Learn Res. 2003;3:993\u20131022.","journal-title":"J Mach Learn Res"},{"key":"576_CR35","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2021.3072209","author":"Z Ma","year":"2021","unstructured":"Ma Z, Lai Y, Xie J, Meng D, Kleijn WB, Guo J, Yu J. Dirichlet process mixture of generalized inverted dirichlet distributions for positive vector data with extended variational inference. IEEE Trans Neural Netw Learn Syst. 2021. https:\/\/doi.org\/10.1109\/TNNLS.2021.3072209.","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"576_CR36","unstructured":"Gregor H. Parameter estimation for text analysis. Technical report (2005)"},{"key":"576_CR37","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1186\/1471-2105-16-S13-S8","volume":"16","author":"W Zhao","year":"2015","unstructured":"Zhao W, Chen JJ, Perkins R, Liu Z, Ge W, Ding Y, Zou W. A heuristic approach to determine an appropriate number of topics in topic modeling. BMC Bioinform. 2015;16:8.","journal-title":"BMC Bioinform"},{"issue":"4","key":"576_CR38","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1145\/99935.99938","volume":"15","author":"F Can","year":"1990","unstructured":"Can F, Ozkarahan EA. Concepts and effectiveness of the cover-coefficient-based clustering methodology for text databases. ACM Trans Database Systems (TODS). 1990;15(4):483\u2013517.","journal-title":"ACM Trans Database Systems (TODS)"},{"key":"576_CR39","unstructured":"Mimno D, Wallach HM, Talley E, Leenders M, McCallum A. Optimizing semantic coherence in topic models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 262\u2013272 (2011). Association for Computational Linguistics."},{"key":"576_CR40","doi-asserted-by":"publisher","first-page":"210","DOI":"10.1515\/crll.1909.136.210","volume":"136","author":"E Hellinger","year":"1909","unstructured":"Hellinger E. Neue begr\u00fcndung der theorie quadratischer formen von unendlichvielen ver\u00e4nderlichen. Journal f\u00fcr die reine und angewandte Mathematik. 1909;136:210\u201371.","journal-title":"Journal f\u00fcr die reine und angewandte Mathematik"},{"key":"576_CR41","volume-title":"Comparing Latent Dirichlet Allocation and latent semantic analysis as classifiers","author":"LH Anaya","year":"2011","unstructured":"Anaya LH. Comparing Latent Dirichlet Allocation and latent semantic analysis as classifiers. Ohio: ERIC; 2011."},{"issue":"1","key":"576_CR42","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1080\/09585190601068508","volume":"18","author":"W Harry","year":"2007","unstructured":"Harry W. Employment creation and localization: the crucial human resource issues for the gcc. Int J Hum Resour Manag. 2007;18(1):132\u201346.","journal-title":"Int J Hum Resour Manag"},{"key":"576_CR43","doi-asserted-by":"crossref","unstructured":"Larose DT. k-nearest neighbor algorithm. Discovering knowledge in data: An introduction to data mining, 90\u2013106 (2005)","DOI":"10.1002\/0471687545.ch5"},{"key":"576_CR44","doi-asserted-by":"crossref","unstructured":"Gatys LA, Ecker AS, Bethge M. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576; 2015.","DOI":"10.1167\/16.12.326"},{"issue":"3","key":"576_CR45","first-page":"385","volume":"3","author":"S Gautam","year":"2018","unstructured":"Gautam S, Soni S. Artificial intelligence techniques for music composition. Int J Sci Res Comput Sci Eng Inform Technol. 2018;3(3):385\u20139.","journal-title":"Int J Sci Res Comput Sci Eng Inform Technol."},{"key":"576_CR46","doi-asserted-by":"crossref","unstructured":"Hansen S, Ramdas T, Sadun R, Fuller J. The demand for executive skills. National Bureau of Economic Research: Technical report; 2021.","DOI":"10.3386\/w28959"}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-022-00576-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40537-022-00576-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-022-00576-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,3,19]],"date-time":"2022-03-19T17:06:18Z","timestamp":1647709578000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-022-00576-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,19]]},"references-count":46,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["576"],"URL":"https:\/\/doi.org\/10.1186\/s40537-022-00576-5","relation":{},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,19]]},"assertion":[{"value":"9 September 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 February 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 March 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This article does not contain any studies with human participants or animals performed by any of the authors.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"32"}}