Towards a Multimodal Model of Cognitive Workload Through Synchronous Optical Brain Imaging and Eye Tracking Measures
- PMID: 31708760
- PMCID: PMC6820355
- DOI: 10.3389/fnhum.2019.00375
Towards a Multimodal Model of Cognitive Workload Through Synchronous Optical Brain Imaging and Eye Tracking Measures
Abstract
Recent advances in neuroimaging technologies have rendered multimodal analysis of operators' cognitive processes in complex task settings and environments increasingly more practical. In this exploratory study, we utilized optical brain imaging and mobile eye tracking technologies to investigate the behavioral and neurophysiological differences among expert and novice operators while they operated a human-machine interface in normal and adverse conditions. In congruence with related work, we observed that experts tended to have lower prefrontal oxygenation and exhibit gaze patterns that are better aligned with the optimal task sequence with shorter fixation durations as compared to novices. These trends reached statistical significance only in the adverse condition where the operators were prompted with an unexpected error message. Comparisons between hemodynamic and gaze measures before and after the error message indicated that experts' neurophysiological response to the error involved a systematic increase in bilateral dorsolateral prefrontal cortex (dlPFC) activity accompanied with an increase in fixation durations, which suggests a shift in their attentional state, possibly from routine process execution to problem detection and resolution. The novices' response was not as strong as that of experts, including a slight increase only in the left dlPFC with a decreasing trend in fixation durations, which is indicative of visual search behavior for possible cues to make sense of the unanticipated situation. A linear discriminant analysis model capitalizing on the covariance structure among hemodynamic and eye movement measures could distinguish experts from novices with 91% accuracy. Despite the small sample size, the performance of the linear discriminant analysis combining eye fixation and dorsolateral oxygenation measures before and after an unexpected event suggests that multimodal approaches may be fruitful for distinguishing novice and expert performance in similar neuroergonomic applications in the field.
Keywords: cognitive workload; eye tracking; fNIRS; human machine interface; neuroergonomics.
Copyright © 2019 İşbilir, Çakır, Acartürk and Tekerek.
Figures
Similar articles
-
How Experts Adapt Their Gaze Behavior When Modeling a Task to Novices.Cogn Sci. 2020 Sep;44(9):e12893. doi: 10.1111/cogs.12893. Cogn Sci. 2020. PMID: 32929803 Free PMC article.
-
Fixation Differences in Visual Search of Accident Scenes by Novices and Expert Emergency Responders.Hum Factors. 2018 Dec;60(8):1219-1227. doi: 10.1177/0018720818788142. Epub 2018 Aug 13. Hum Factors. 2018. PMID: 30102566
-
Mental workload assessment by monitoring brain, heart, and eye with six biomedical modalities during six cognitive tasks.Front Neuroergon. 2024 Mar 12;5:1345507. doi: 10.3389/fnrgo.2024.1345507. eCollection 2024. Front Neuroergon. 2024. PMID: 38533517 Free PMC article.
-
From gaze cueing to dual eye-tracking: novel approaches to investigate the neural correlates of gaze in social interaction.Neurosci Biobehav Rev. 2013 Dec;37(10 Pt 2):2516-28. doi: 10.1016/j.neubiorev.2013.07.017. Epub 2013 Aug 5. Neurosci Biobehav Rev. 2013. PMID: 23928088 Review.
-
In Vivo Observations of Rapid Scattered Light Changes Associated with Neurophysiological Activity.In: Frostig RD, editor. In Vivo Optical Imaging of Brain Function. 2nd edition. Boca Raton (FL): CRC Press/Taylor & Francis; 2009. Chapter 5. In: Frostig RD, editor. In Vivo Optical Imaging of Brain Function. 2nd edition. Boca Raton (FL): CRC Press/Taylor & Francis; 2009. Chapter 5. PMID: 26844322 Free Books & Documents. Review.
Cited by
-
Exploring cognitive load through neuropsychological features: an analysis using fNIRS-eye tracking.Med Biol Eng Comput. 2024 Aug 6. doi: 10.1007/s11517-024-03178-w. Online ahead of print. Med Biol Eng Comput. 2024. PMID: 39107650
-
A systematic review of neurophysiological sensing for the assessment of acute pain.NPJ Digit Med. 2023 Apr 26;6(1):76. doi: 10.1038/s41746-023-00810-1. NPJ Digit Med. 2023. PMID: 37100924 Free PMC article. Review.
-
Using Task-Evoked Pupillary Response to Predict Clinical Performance during a Simulation Training.Healthcare (Basel). 2023 Feb 4;11(4):455. doi: 10.3390/healthcare11040455. Healthcare (Basel). 2023. PMID: 36832990 Free PMC article.
-
Programming ability prediction: Applying an attention-based convolutional neural network to functional near-infrared spectroscopy analyses of working memory.Front Neurosci. 2022 Dec 1;16:1058609. doi: 10.3389/fnins.2022.1058609. eCollection 2022. Front Neurosci. 2022. PMID: 36532289 Free PMC article.
-
Depression and Cognitive Impairment: Current Understanding of Its Neurobiology and Diagnosis.Neuropsychiatr Dis Treat. 2022 Nov 29;18:2783-2794. doi: 10.2147/NDT.S383093. eCollection 2022. Neuropsychiatr Dis Treat. 2022. PMID: 36471744 Free PMC article. Review.
References
-
- Ayaz H. (2010). Functional Near Infrared Spectroscopy Based Brain Computer Interface. Philadelphia, PA: Drexel University.
-
- Ayaz H., Izzetoglu M., Platek S. M., Bunce S., Izzetoglu K., Pourrezaei K., et al. (2006). “Registering fNIR data to brain surface image using MRI templates,” in Proceedings of the 2006 International Conference of the IEEE Engineering in Medicine and Biology Society (New York, NY: IEEE; ), 2671–2674. - PubMed
-
- Ayaz H., Onaral B., Izzetoglu K., Shewokis P. A., McKendrick R., Parasuraman R. (2013). Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development. Front. Hum. Neurosci. 7:871. 10.3389/fnhum.2013.00871 - DOI - PMC - PubMed
LinkOut - more resources
Full Text Sources