Abstract
Adaptive automation shall support users in a flexible way. One way to achieve this could be by monitoring cognitive states of pilots in order to anticipate an individual’s need for support. A special challenge lies in choosing methods that enable a valid measurement of the cognitive state in question since different measures are associated with distinct strengths and weaknesses. For example, practical considerations like environmental factors, wearing comfort and intrusiveness have to be considered. The objective of this paper is to provide a collection of physiological, self-report and behavioral measures that can be applied to assess mental workload in pilots, and to discuss their advantages and disadvantages for this purpose. A targeted literature search was conducted to this end. The comparisons drawn in this paper reveal that a multi-method approach is preferable to relying on a single measure. In this regard, however, there is no one-size-fits-all solution and it is strongly advised to consider the selection of appropriate measures carefully for each specific research question and application context.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wickens, C.D., Dehais, F.: Expertise in aviation. In: Ward, P., Maarten Schraagen, J., Gore, J., Roth, E.M. (eds.) The Oxford Handbook of Expertise. Oxford University Press, Oxford (2019)
Bainbridge, L.: Ironies of automation. In: Johannsen, G., Rijnsdorp, J.E. (eds.) Analysis, Design and Evaluation of Man–Machine Systems, Pergamon, pp. 129–135 (1983)
Billings, C.E.: Human-centered aviation automation: principles and guidelines. In. Ames Research Center, Moffett Field, California (1996)
Parasuraman, R.M., Mouloua, M., Hilburn, B.: adaptive aiding and adaptive task allocation enhance human-machine interaction. In: Paper Presented at the Automation Technology and Human Performance: Current Research and Trends, Norfolk, VA (1999)
Martins, A.P.G.: A review of important cognitive concepts in aviation. Aviation 20(2), 65–84 (2016). https://doi.org/10.3846/16487788.2016.1196559
Charles, R.L., Nixon, J.: Measuring mental workload using physiological measures: a systematic review. Appl. Ergon. 74, 221–232 (2019). https://doi.org/10.1016/j.apergo.2018.08.028
Edwards, T.: Human performance in air traffic control. University of Nottingham (2013)
Parasuraman, R., Sheridan, T.B., Wickens, C.D.: Situation awareness, mental workload, and trust in automation: viable, empirically supported cognitive engineering constructs. J. Cogn. Eng. Decis. Mak. 2(2), 140–160 (2008). https://doi.org/10.1518/155534308x284417
Dekker, S., Hollnagel, E.: Human factors and folk models. Cogn. Technol. Work 6(2), 79–86 (2004). https://doi.org/10.1007/s10111-003-0136-9
O'Donnell, R.D., Eggemeier, F.T.: Workload assessment methodology. In: Boff, K.R., Kaufman, L., Thomas, J.P. (eds.) Handbook of Perception and Human Performance. John Wiley & Sons, New York (1986)
Stokes, A., Kite, K.: Flight stress: stress, fatigue, and performance in aviation. Repr ed. Avebury, Aldershot (1997)
Fürstenau, N., Papenfuss, A.: Model based analysis of subjective mental workload during multiple remote tower human-in-the-loop simulations. In: Fürstenau, N. (ed.) Virtual and Remote Control Tower: Research, Design, Development, Validation, and Implementation, pp. 293–342. Springer, Cham (2022)
Liu, H., Allen, J., Zheng, D., Chen, F.: Recent development of respiratory rate measurement technologies. Physiol. Meas. 40(7), 1–27 (2019). https://doi.org/10.1088/1361-6579/ab299e
Roscoe, A.H.: Assessing pilot workload: Why measure heart rate, HRV and respiration. Biol. Psychol. 34(2), 259–287 (1992). https://doi.org/10.1016/0301-0511(92)90018-P
AL-Khalidi, F.Q., Saatchi, R., Burke, D., Elphick, H., Tan, S.: Respiration rate monitoring methods: a review. Pediat. Pulmonol. 46(6), 523–529 (2011). https://doi.org/10.1002/ppul.21416
Brookings, J.B., Wilson, G.F., Swain, C.R.: Psychophysiological responses to changes in workload during simulated air traffic control. Biol. Psychol. 42(3), 361–377 (1996). https://doi.org/10.1016/0301-0511(95)05167-8
Mehler, B., Reimer, B., Coughlin, J.F., Dusek, J.A.: Impact of incremental increases in cognitive workload on physiological arousal and performance in young adult drivers. Transp. Res. Rec. 2138(1), 6–12 (2009). https://doi.org/10.3141/2138-02
Fairclough, S.H., Venables, L., Tattersall, A.: The influence of task demand and learning on the psychophysiological response. Int. J. Psychophysiol. 56(2), 171–184 (2005). https://doi.org/10.1016/j.ijpsycho.2004.11.003
Backs, R.W., Navidzadeh, H.T., Xu, X.: Cardiorespiratory indices of mental workload during simulated air traffic control. Proc. Hum. Fact. Ergon. Soc. Ann. Meet. 44(13), 89–92 (2000). https://doi.org/10.1177/154193120004401323
Backs, R.W., Seljos, K.A.: Metabolic and cardiorespiratory measures of mental effort: the effects of level of difficulty in a working memory task. Int. J. Psychophysiol. 16(1), 57–68 (1994). https://doi.org/10.1016/0167-8760(94)90042-6
Fairclough, S.H., Venables, L.: Prediction of subjective states from psychophysiology: a multivariate approach. Biol. Psychol. 71(1), 100–110 (2006). https://doi.org/10.1016/j.biopsycho.2005.03.007
Hogervorst, M.A., Brouwer, A.-M., van Erp, J.B.F.: Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload. Front. Neurosci. 8 (2014). https://doi.org/10.3389/fnins.2014.00322
Fournier, L.R., Wilson, G.F., Swain, C.R.: Electrophysiological, behavioral, and subjective indexes of workload when performing multiple tasks: manipulations of task difficulty and training. Int. J. Psychophysiol. 31(2), 129–145 (1999). https://doi.org/10.1016/S0167-8760(98)00049-X
Berntson, G.G., Quigley, K.S., Norman, G.J., Lozano, D.L.: Cardiovascular psychophysiology. In: Handbook of Psychophysiology. Cambridge Handbooks in Psychology, 4 edn, pp. 183–216. Cambridge University Press, New York (2017)
Ayada, C., Toru, Ü., Korkut, Y.: The relationship of stress and blood pressure effectors. Hippokratia 19(2), 99–108 (2015)
Charkoudian, N., Rabbitts, J.A.: Sympathetic neural mechanisms in human cardiovascular health and disease. Mayo Clin. Proc. 84(9), 822–830 (2009). https://doi.org/10.4065/84.9.822
Lundberg, U., et al.: Psychophysiological stress and EMG activity of the trapezius muscle. Int. J. Behav. Med. 1(4), 354–370 (1994). https://doi.org/10.1207/s15327558ijbm0104_5
Veltman, J.A., Gaillard, A.W.K.: Physiological workload reactions to increasing levels of task difficulty. Ergonomics 41(5), 656–669 (1998). https://doi.org/10.1080/001401398186829
Veltman, J.A., Gaillard, A.W.K.: Physiological indices of workload in a simulated flight task. Biol. Psychol. 42(3), 323–342 (1996). https://doi.org/10.1016/0301-0511(95)05165-1
James, G.D., Gerber, L.M.: Measuring arterial blood pressure in humans: auscultatory and automatic measurement techniques for human biological field studies. Am. J. Hum. Biol. 30(1), e23063 (2018). https://doi.org/10.1002/ajhb.23063
Xing, X., et al.: Blood pressure assessment with in-ear photoplethysmography. Physiol. Meas. 42(10), 105009 (2021). https://doi.org/10.1088/1361-6579/ac2a71
Braithwaite, J.J.W., Jones, D.G., Rowe, M.R.: A guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psychological experiments. In: Behavioural Brain Sciences Centre, p. 43. University of Birmingham, UK (2015)
Boucsein, W., et al.: Society for psychophysiological research ad hoc committee on electrodermal measures: publication recommendations for electrodermal measurements. Psychophysiology 49(8), 1017–1034 (2012). https://doi.org/10.1111/j.1469-8986.2012.01384.x
Jackson, A.F., Bolger, D.J.: The neurophysiological bases of EEG and EEG measurement: a review for the rest of us. Psychophysiology 51(11), 1061–1071 (2014). https://doi.org/10.1111/psyp.12283
Radüntz, T.: Signal quality evaluation of emerging EEG devices. Front. Physiol. 9, 98 (2018). https://doi.org/10.3389/fphys.2018.00098
Wascher, E., et al.: Neuroergonomics on the go: An evaluation of the potential of mobile EEG for workplace assessment and design. Hum. Fact., 187208211007707 (2021). https://doi.org/10.1177/00187208211007707
Dussault, C., Jouanin, J.-C., Guezennec, C.-Y.: EEG and ECG changes during selected flight sequences. Aviat. Space Environ. Med. 75(10), 889–897 (2004)
Hamann, A., Carstengerdes, N.: Investigating mental workload-induced changes in cortical oxygenation and frontal theta activity during simulated flights. Sci. Rep. 12(1), 6449 (2022). https://doi.org/10.1038/s41598-022-10044-y
Puma, S., Matton, N., Paubel, P.-V., Raufaste, É., El-Yagoubi, R.: Using theta and alpha band power to assess cognitive workload in multitasking environments. Int. J. Psychophysiol. Off. J. Int. Organ. Psychophysiol. 123, 111–120 (2018). https://doi.org/10.1016/j.ijpsycho.2017.10.004
Dehais, F., et al.: Monitoring pilot’s mental workload using ERPs and spectral power with a six-dry-electrode EEG system in real flight conditions. Sensors (Basel, Switzerland) 19(6), 1324 (2019). https://doi.org/10.3390/s19061324
Holm, A., Lukander, K., Korpela, J., Sallinen, M., Müller, K.M.I.: Estimating brain load from the EEG. Sci. World J. 9, 639–651 (2009). https://doi.org/10.1100/tsw.2009.83
Roy, R.N., Bonnet, S., Charbonnier, S., Campagne, A.: Mental fatigue and working memory load estimation: interaction and implications for EEG-based passive BCI. In: Conference Proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6607–6610 (2013). https://doi.org/10.1109/embc.2013.6611070
Radüntz, T., Meffert, B.: User experience of 7 mobile electroencephalography devices: comparative study. JMIR Mhealth Uhealth 7(9), e14474 (2019). https://doi.org/10.2196/14474
Huppert, T.J., Franceschini, M.A., Boas, D.A.: Noninvasive imaging of cerebral activation with diffuse optical tomography. In: Frostig, R. (ed.) In Vivo Optical Imaging of Brain Function, pp. 209–238. CRC Press (2009)
Huppert, T.J.: Commentary on the statistical properties of noise and its implication on general linear models in functional near-infrared spectroscopy. Neurophotonics 3(1), 010401 (2016). https://doi.org/10.1117/1.NPh.3.1.010401
Liu, T., Pelowski, M., Pang, C., Zhou, Y., Cai, J.: Near-infrared spectroscopy as a tool for driving research. Ergonomics 59(3), 368–379 (2015). https://doi.org/10.1080/00140139.2015.1076057
Barker, J.W., Aarabi, A., Huppert, T.J.: Autoregressive model based algorithm for correcting motion and serially correlated errors in fNIRS. Biomed. Opt. Express 4(8), 1366–1379 (2013). https://doi.org/10.1364/boe.4.001366
Causse, M., Chua, Z.K., Peysakhovich, V., Del Campo, N., Matton, N.: Mental workload and neural efficiency quantified in the prefrontal cortex using fNIRS. Sci. Rep. 7(1), 5222 (2017). https://doi.org/10.1038/s41598-017-05378-x
Geissler, C.F., Schneider, J., Frings, C.: Shedding light on the prefrontal correlates of mental workload in simulated driving: a functional near-infrared spectroscopy study. Sci. Rep. 11(1), 705 (2021). https://doi.org/10.1038/s41598-020-80477-w
Causse, M., Chua, Z.K., Rémy, F.: Influences of age, mental workload, and flight experience on cognitive performance and prefrontal activity in private pilots: a fNIRS study. Sci. Rep. 9(1), 7688 (2019). https://doi.org/10.1038/s41598-019-44082-w
Nguyen, T., Ahn, S., Jang, H., Jun, S.C., Kim, J.G.: Utilization of a combined EEG/NIRS system to predict driver drowsiness. Sci. Rep. 7, 43933 (2017). https://doi.org/10.1038/srep43933
Sammito, S., Thielmann, B., Klussmann, A., Deußen, A., Braumann, K.-M., Böckelmann, I.: S2k-Leitlinie Nutzung der Herzschlagfrequenz und der Herzfrequenzvariabilität in der Arbeitsmedizin und der Arbeitswissenschaft: AWMF-RegNr 002/042 (2021)
Kingsley, M., Lewis, M.J., Marson, R.E.: Comparison of Polar 810s and an ambulatory ECG system for RR interval measurement during progressive exercise. Int. J. Sports Med. 26(1), 39–44 (2005). https://doi.org/10.1055/s-2004-817878
Nunan, D., Jakovljevic, D.G., Donovan, G., Hodges, L.D., Sandercock, G.R., Brodie, D.A.: Levels of agreement for RR intervals and short-term heart rate variability obtained from the Polar S810 and an alternative system. Eur. J. Appl. Physiol. 103(5), 529–537 (2008). https://doi.org/10.1007/s00421-008-0742-6
Radespiel-Troger, M., Rauh, R., Mahlke, C., Gottschalk, T., Muck-Weymann, M.: Agreement of two different methods for measurement of heart rate variability. Clin. Auton. Res. 13(2), 99–102 (2003). https://doi.org/10.1007/s10286-003-0085-7
Treiber, F.A., Musante, L., Hartdagan, S., Davis, H., Levy, M., Strong, W.B.: Validation of a heart rate monitor with children in laboratory and field settings. Med. Sci. Sports Exerc. 21(3), 338–342 (1989)
Hettinger, T., Wobbe, G.: Kompendium der Arbeitswissenschaft. Kiehl Verlag, Ludwigshafen (1993)
Finsen, L., Søgaard, K., Jensen, C., Borg, V., Christensen, H.: Muscle activity and cardiovascular response during computer-mouse work with and without memory demands. Ergonomics 44(14), 1312–1329 (2001). https://doi.org/10.1080/00140130110099065
Malik, M., et al.: Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Eur. Heart J. 17(3), 354–381 (1996). https://doi.org/10.1093/oxfordjournals.eurheartj.a014868
Nunan, D., Sandercock, G.R.H., Brodie, D.: A quantitative systematic review of normal values for short-term heart rate variability in healthy adults. Pace-Pacing Clin. Electrophysiol. 33(11), 1407–1417 (2010). https://doi.org/10.1111/j.1540-8159.2010.02841.x
Schafer, A., Vagedes, J.: How accurate is pulse rate variability as an estimate of heart rate variability? a review on studies comparing photoplethysmographic technology with an electrocardiogram. Int. J. Cardiol. 166(1), 15–29 (2013). https://doi.org/10.1016/j.ijcard.2012.03.119
Lohani, M., Payne, B.R., Strayer, D.L.: A review of psychophysiological measures to assess cognitive states in real-world driving. Front. Hum. Neurosci. 13, 57 (2019). https://doi.org/10.3389/fnhum.2019.00057
Sammito, S., Bockelmann, I.: Analysis of heart rate variability: mathematical description and practical application. Herz 40, 76–84 (2015). https://doi.org/10.1007/s00059-014-4145-7
Duchowski, A.T.: Eye Tracking Methodology: Theory and Practice, 3rd edn. Springer, Heidelberg (2007)
Goldberg, J., Wichansky, A.: Eye tracking in usability evaluation: A practitioner's guide. (2003)
Glaholt, M.G.: Eye tracking in the cockpit: a review of the relationships between eye movements and the aviator’s cognitive state. In. Defence Research and Development Canada, Toronto, Canada (2014)
Tole, J.R., Stephens, A.T., Vivaudou, M., Ephrath, A.R., Young, L.R.: Visual scanning behavior and pilot workload. In. NASA (1983)
Faulhaber, A.K., Friedrich, M.: Eye-tracking metrics as an indicator of workload in commercial single-pilot operations. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2019. CCIS, vol. 1107, pp. 213–225. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32423-0_14
Di Nocera, F., Camilli, M., Terenzi, M.: A random glance at the flight deck: pilots’ scanning strategies and the real-time assessment of mental workload. J. Cogn. Eng. Dec. Mak. 1(3), 271–285 (2007). https://doi.org/10.1518/155534307X255627
Stanton, N.A., Salmon, P.M., Rafferty, L.A., Walker, G.H., Baber, C., Jenkins, D.P.: Human Factors Methods: A Practical Guide for Engineering and Design, 2nd edn. CRC Press, London (2013)
Wierwille, W.W., Eggemeier, F.T.: Recommendations for mental workload measurement in a test and evaluation environment. Hum. Factors 35(2), 263–281 (1993). https://doi.org/10.1177/001872089303500205
Wickens, C.D.: Multiple resources and performance prediction. Theor. Issues Ergon. Sci. 3(2), 159–177 (2002). https://doi.org/10.1080/14639220210123806
Wickens, C.D.: Multiple resources and mental workload. Hum. Factors 50(3), 449–455 (2008). https://doi.org/10.1518/001872008x288394
Casali, J.G., Wierwille, W.W.: A comparison of rating scale, secondary-task, physiological, and primary-task workload estimation techniques in a simulated flight task emphasizing communications load. Hum. Factors 25(6), 623–641 (1983). https://doi.org/10.1177/001872088302500602
Casali, J.G., Wierwille, W.W.: On the measurement of pilot perceptual workload: a comparison of assessment techniques addressing sensitivity and intrusion issues. Ergonomics 27(10), 1033–1050 (1984). https://doi.org/10.1080/00140138408963584
Wierwille, W.W., Connor, S.A.: Evaluation of 20 workload measures using a psychomotor task in a moving-base aircraft simulator. Hum. Factors 25(1), 1–16 (1983). https://doi.org/10.1177/001872088302500101
Hart, S.G.: Nasa-task load index (NASA-TLX); 20 years later. Proc. Hum. Fact. Ergon. Soc. Ann. Meet. 50(9), 904–908 (2006). https://doi.org/10.1177/154193120605000909
Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Advances in Psychology, North-Holland, vol. 52. pp. 139–183 (1988)
Tattersall, A.J., Foord, P.S.: An experimental evaluation of instantaneous self-assessment as a measure of workload. Ergonomics 39(5), 740–748 (1996). https://doi.org/10.1080/00140139608964495
Zimeo Morais, G.A., Balardin, J.B., Sato, J.R.: fNIRS optodes’ location decider (fOLD): a toolbox for probe arrangement guided by brain regions-of-interest. Sci Rep 8(1), 3341 (2018). https://doi.org/10.1038/s41598-018-21716-z
Biella, M., Wies, M.: Human performance envelope: overview of the project and technical results. In: Paper presented at the Future Sky Safety Final Conference, FSS on Final Approach, Brüssel, Belgium (2018)
Acknowledgements
DLR’s research work was financed with funding from the German Federal Ministry of Defence. Stefan Sammito is an active Bundeswehr Medical Service officer and works for the German Federal Ministry of Defence. This paper reflects the opinion of the authors and not necessarily the opinion of the German Federal Ministry of Defence or the Surgeon General of the German Air Force.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Boumann, H., Hamann, A., Biella, M., Carstengerdes, N., Sammito, S. (2023). Suitability of Physiological, Self-report and Behavioral Measures for Assessing Mental Workload in Pilots. In: Harris, D., Li, WC. (eds) Engineering Psychology and Cognitive Ergonomics. HCII 2023. Lecture Notes in Computer Science(), vol 14017. Springer, Cham. https://doi.org/10.1007/978-3-031-35392-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-35392-5_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35391-8
Online ISBN: 978-3-031-35392-5
eBook Packages: Computer ScienceComputer Science (R0)