{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T15:06:03Z","timestamp":1729177563365,"version":"3.27.0"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643685489","type":"electronic"}],"license":[{"start":{"date-parts":[[2024,10,16]],"date-time":"2024-10-16T00:00:00Z","timestamp":1729036800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,10,16]]},"abstract":"Many industrial sectors rely on well-trained employees that are able to operate complex machinery. In this work, we demonstrate an immersive assistance system powered by Artificial Intelligence (AI) that supports users in performing complex tasks in industrial environments. Our system leverages a Virtual Reality (VR) environment that resembles a juice mixer setup. This digital twin of a physical setup simulates complex industrial machinery used to mix preparations or liquids (e.g., similar to the pharmaceutical industry) and includes various containers, sensors, pumps, and flow controllers. This setup demonstrates our system\u2019s capabilities in a controlled environment while acting as a proof-of-concept for broader industrial applications. The core components of our multimodal AI assistant are a large language model and a speech-to-text model that process a video and audio recording of an expert performing the task in a VR environment. The video and speech input extracted from the expert\u2019s video enables it to provide step-by-step guidance to support users in executing complex tasks. This demonstration showcases the potential of our AI-powered assistant to reduce cognitive load, increase productivity, and enhance safety in industrial environments.<\/jats:p>","DOI":"10.3233\/faia241037","type":"book-chapter","created":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T13:58:59Z","timestamp":1729173539000},"source":"Crossref","is-referenced-by-count":0,"title":["AI-Powered Immersive Assistance for Interactive Task Execution in Industrial Environments"],"prefix":"10.3233","author":[{"given":"Tomislav","family":"Duricic","sequence":"first","affiliation":[{"name":"Know-Center GmbH"},{"name":"Graz University of Technology"}]},{"given":"Peter","family":"M\u00fcllner","sequence":"additional","affiliation":[{"name":"Know-Center GmbH"},{"name":"Graz University of Technology"}]},{"given":"Nicole","family":"Weidinger","sequence":"additional","affiliation":[{"name":"Know-Center GmbH"}]},{"given":"Neven","family":"ElSayed","sequence":"additional","affiliation":[{"name":"Know-Center GmbH"}]},{"given":"Dominik","family":"Kowald","sequence":"additional","affiliation":[{"name":"Know-Center GmbH"},{"name":"Graz University of Technology"}]},{"given":"Eduardo","family":"Veas","sequence":"additional","affiliation":[{"name":"Know-Center GmbH"},{"name":"Graz University of Technology"}]}],"member":"7437","container-title":["Frontiers in Artificial Intelligence and Applications","ECAI 2024"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/FAIA241037","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T13:59:00Z","timestamp":1729173540000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/FAIA241037"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,16]]},"ISBN":["9781643685489"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/faia241037","relation":{},"ISSN":["0922-6389","1879-8314"],"issn-type":[{"value":"0922-6389","type":"print"},{"value":"1879-8314","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,16]]}}}