Keywords

1 Introduction

The ability to conceive of the aircraft’s whereabouts, status, weather, fuel state, terrain, and, in combat, enemy disposition is critical to effective aircraft operation. In critical phases of flight, in poor weather, or in the face of systems malfunctions, this ability can mean the difference between mission success and failure or even aircrew survivability [1]. Operator performance in complex or dynamic environments is often a function of situation awareness (SA) [2]. Aircrews spend considerable portions of time and effort developing and maintaining SA, especially in evolving environments [3]. Indeed the military has had a keen interest in SA in the cockpit dating back to World War I and especially more so in the information age of the late 20th century [4–6]. In tactical situations, this includes knowledge about the locations, actions, and capabilities of both friendly and enemy forces [3]. A leading cause of military and 88 % of commercial aviation accidents have been attributed to poor SA [7, 8]. If SA is essential to mission accomplishment, then harnessing the pilot interaction with the cockpit controls and displays which aid the collection of key information is a driving goal in both engineering design and operator training. This paper explores the operator’s perspective on how cockpit tasks affect SA.

2 Background

Situation awareness (SA) is the knowledge of environmental factors that influence decisions and depends on the operator’s internal perceptual model of the world [9]. More specifically, SA has been defined as the operator’s perception (or mental model) of elements in the environment around them within a volume of space and time, the comprehension of their meaning, and the projection of their status into the future [3, 10]. Non-routine and unpredictable situations, such as those that can occur during emergency situations or combat, demand effective integration of large quantities of information to be processed with limited cognitive capacity [2, 4]. Situation awareness drives decision making based on the operator’s mental model of the environment. The process of gaining situation awareness, while independent from the process of decision making, forms a basis for this mental model [4, 11]. The performance of a pilot is highly related to their ability to fuse information to augment SA and drive decision-making [12]. The resulting decision leads to an action, which alters the environment and situation and the cycle begins again.

No current criteria exist to determine the required level of SA needed to obtain a desired level of performance [13]. Less than perfect SA causes the operator to assume a level of risk for error as they must make a decision without complete information. Good SA increases the probability of good performance, but does not guarantee high performance as factors, such as excessive workload, poor decision making, or poor task execution can degrade performance in the presence of good SA [11]. Because of the critical effects situation awareness has on mission outcome and survivability, designers have a vested interest in providing the operator with appropriate resources, and operators have a vested interest in maintaining a high level of SA [1].

Designers can incorporate heads-up displays (HUD), multi-function displays (MFD), automation, expert systems, advanced avionics, and sensors to provide more information in a more useful manner [10]. In this regard, for a given task, all cockpits are not created equal. Cockpit design affects the number of required tasks the operator must perform, the workload of those tasks, and the information provided to the operator during their completion [4]. Automation and intelligent cockpits can tailor information to match the operator’s needs or perform support functions based on the required situation [2]. Operators develop procedures to scan information sources and train to maximize the use of all available tools and observations to increase SA.

Traditional methods for measuring SA often probe the operator’s awareness of critical facts within an environment or require the operator or an observer to rate the operator’s SA when using a system or simulator [14]. However, these methods do not readily provide insight into the utility of any single device in the cockpit for improving SA. Evaluations of the impact of individual display or display formats on SA are limited within the human factors research, but often include evaluating a system in which a single display is modified within the cockpit [15].

It might be reasonable to assume that by increasing the availability of information in the cockpit, SA will be improved. However, the literature and experience demonstrate that this assumption is not true. For example:

  • Cues in the normal field of view can be missed if they are too subtle or if the operator is not paying sufficient attention [16], yet most pilots would immediately perceive a flashing “Master Warning” light, especially if it was accompanied by an audible warning tone. However, frequent or nuisance warnings desensitize operators, thus negating the effectiveness of those warnings [17, 18].

  • As cockpits evolve, new “black boxes” are added: autopilot, radar, weapons systems, mission sensors, navigational aids, computers, etc. While many of the additions undeniably present more information to the aircrew and have the potential to solve specific problems, they each present another interface to the human operator, increasing workload and decreasing the time available to scan each device.

  • Information overload is a significant concern in modern cockpits [2, 19]. SA is inherently a function of the quantity and quality of data, however the format of image presentation determines the availability and the likelihood of use of that data. Poorly designed and integrated interfaces increase workload and time needed to synthesize information and open opportunities for confusion and errors [2]. Conversely, well-designed interfaces can reduce workload. However, even the best cockpit displays and controls cannot overcome human performance limits [20], including the human operator’s capacity for attention of 2−40 bits per second; far lower than the presentation rate of the environment and modern cockpit systems [21].

  • Because glass cockpits rely much more on integrating display layers and inputs, primarily through a Flight Management System (FMS). These systems provide more data in a richer format. However, they also require paging through multiple display layers. As such, interaction now requires a keyboard for instrument manipulation, therefore, accessing a function that once required twisting a dial or a couple of switches is more complex—with an associated increase in workload and time delay.

  • By reducing the number of places a pilot is required to scan within the cockpit, attention resources available to the remaining scan areas can be increased. However, if the pilot neglects to maintain a scan elsewhere and becomes dependent or fixated on certain information sources (e.g., the HUD), his or her SA will remain limited [22–24]. Dependency on automation elements may cause fixation to the neglect of other duties.

  • Automation can increase feedback or provide new methods of communicating feedback, which may alter how an operator must assimilate that data [25]. If performed well, this can aid the operator in comprehending the meaning of items in the environment and projecting future states; however, if performed poorly, critical information may be obscured.

Each of these examples illustrates that each information source in the cockpit may have a significant influence (positive or negative) on SA. Therefore, there is a need for a method to allow one to understand the effect of each of these information sources on the operator’s SA, allowing one to assess the importance of each display and perhaps each information representation. As indicated before, however, the presence of a display does not aid SA. Instead SA is gained as the operator actively interacts with the system, gaining information through this interaction. Since the cockpit and situation awareness are inextricably linked, this paper seeks to answer the question “how do various cockpit tasks contribute to aircrew SA?”

3 Methodology

Meyer [1] distilled over 1200 different cockpit tasks found in typical C-130 airlift missions into 19 generic cockpit tasks representing human-machine interaction and non-machine interaction tasks. This list of tasks is provided in Table 1. These generic tasks can easily be applied as an analogy to the majority of cockpit tasks found in any aircraft. These generic tasks can then be classified as machine or non-machine interaction tasks; where the human either interacts with the machine or with other humans. Some generic tasks, specifically talking and listening tasks were difficult to classify. For instance, if these tasks were conducted via radio or aircraft but the communication was human-human interaction with electronic transmission means, the tasks were classified as non-machine interaction tasks. However, aircraft generated advisories (listening, simple) and monitoring the Radar Warning Receiver (RWR), which permit interaction between the human and the machine could be classified as machine interaction tasks. However, they generally require natural interfaces as would be employed when interacting with another human and are therefore classified as non-machine interaction tasks.

Table 1. Generic cockpit tasks
Full size table

In order to quantify the relative contributions of each individual task to tactical SA, a survey was given to 21 aviators representing pilots and non-pilot operators of various military aircraft. The number of respondents in each aircraft type and crew position are shown in Table 2. Several of the participants had flown on multiple aircraft (hence aircraft count is greater than 21). C-130 aircrew and pilots were intentionally over-represented because Meyer [1] originally focused on the C-130 aircraft.

Table 2. Survey representation
Full size table

Respondents were asked to rate the relative impact of each task on SA. When performing this rating, participants were asked to provide a rating between −3 and 3. Participants were to provide this rating against a hypothetical baseline of flying with one’s eyes closed, which was to receive a rating of 0. Note that negative numbers indicate that the tasks detract from SA, zero equals no effect of a task on SA, and positive values indicate that performing the task contributes to SA. Participants were asked to disregard the effects of workload on their SA scores and provide any remarks that they felt were appropriate. Because it is expected that SA demands change as a function of mission phase, these ratings were made for three different mission phases: cruise flight, formation airdrop, and a maximum effort landing.

4 Results

Since not every survey participant was familiar with every mission phase (i.e. F-15E pilots do not perform airdrop), or are in suitably equipped aircraft, each task did not necessarily receive 21 responses. Most notably, tasks involving the head’s-up display (HUD) during the airdrop and maximum effort landing mission phases was limited to C-130 J and MC-130 J participants (n = 4). However, the standard deviation for the HUD tasks (σ = 0.52, 0.96, 0.00) was normal compared to standard deviations of other tasks with 14-21 respondents. Indeed standard deviations were relatively large due to the integer scoring and small sample size (σaverage = 1.33). Noticeably, tasks that induced positive SA tended to have lower standard deviations than negative SA tasks as shown in Table 4.

By far, the strongest sentiment in participant feedback was, “it depends.” What is clearly evident from Tables 3 and 4 is that the situation itself is a strong independent variable. Objective area events, such as airdrop and airland, produced slightly stronger opinions than in cruise flight. Tasks that related to visual cues, such as looking out of the window or HUD have stronger SA scores when the aircraft was lower to the ground and when those visual cues provide richer and more detailed information. The more cognitively demanding tasks of keyboard entry, complex talking (i.e. giving a briefing), writing, and manual computations produced stronger SA penalties during objective area events. Human-machine interaction tasks that provided strong positive SA tended to receive even higher ratings during objective area events; non-machine interaction tasks that incurred strong SA penalties received lower ratings during objective area events (i.e. events near targets, drop zones, and landing zones). Conversely, human-machine interaction SA penalties did not get appreciably worse during objective area events (except keyboard entry), while non-machine interaction tasks with strong SA scores did not improve noticeably during objective area events (except for looking out of the window).

Table 3. SA scores, human-machine interaction tasks
Full size table
Table 4. SA scores, non-machine interaction tasks
Full size table

When viewing human-machine interaction tasks (Table 3), a strong preference emerges from four tasks: viewing the HUD, reading a multifunction display (MFD) or other similar moving map or digital display, reading instruments and gauges, and radar/sensor interpretation. The HUD scored moderately well during cruise flight, but the SA value increased remarkably during airdrop and airland, receiving the only scores above 2.0. Clearly the relevance of HUD information, whether by design or its relevance, increases dramatically during these two objective area events where aircraft are lower to the ground. The MFD SA averages were consistently high across all phases of flight. The quantity, quality, and relevance of this data does not seem to depend on phase of flight as much as it depends on crew position (flight engineers and some navigators consistently gave it lower SA scores, while pilots gave considerably higher SA scores to the MFD).

Surprisingly, instrument/gauge reading and radar/sensor interpretation did not exhibit clearly defined trends when viewed as averages. Reading an instrument infers reading a single component and gaining few elements of information. Compared to digital displays such as a HUD or MFD (which superimpose multiple data sources into the pilot’s field of view or onto a moving map display), the gauges–by design–present far less data. However, it is surprising that in cruise flight, the SA value of raw instrument reading is nearly identical to viewing the HUD. Furthermore, raw instrument data scored lowest during airdrop and highest during airland. These trends can partially be explained when considering the non-C-130 aircrew did not answer the airdrop and airland portions. Pilots had stronger opinions of the value of instrument reading than navigators (and even some flight engineers), and pilots made up the bulk of non-C-130 participants that were represented only in the cruise flight category. This same trend was also partially evident in the radar/sensor interpretation task, but several participants indicated that it was of less value during the objective area than cruise flight. It is unclear if this is caused by radar/sensor information being degraded at low altitude, the information having less relevance during those phases, or a preference for other tasks, such as looking outside the window or HUD.

Similar to the HUD values shown in Table 3, looking out of the window increased in value during objective area events as shown in Table 4. In cruise flight, the window had similar SA values to the HUD, despite the HUD providing extra data. While pilots naturally have a strong SA value derived from looking outside, those situated farther from the cockpit windows did not feel as strongly. Naturally, the one respondent who stated, “I can barely see out the window from where I sit,” rated the value of looking out of the window as marginal. Using this task as an example, one pilot emphasized that not only are phases of flight an independent variable, but specific situational elements add a host of other variables. For example, when looking out of the window, “night or low illumination will lower SA. If flying in formation through the weather, SA goes down, but looking through the window in daylight with good visibility in the mountains will greatly increase SA.”

Naturally, reading a chart, map, or approach diagram will provide the operator with SA. Participants also showed a preference for simple listening tasks (i.e. “too low” or “5 knots fast”) over more complex listening. Complex listening provided more SA during cruise flight than during objective area operations, which reinforces operational practices to conduct crew briefings during cruise portions of flight where possible. However, it is indeterminate if this trend has to do with typically lower attentional demands during cruise that allows for more complex listening, or because cruise flight grants more relevance, quantity, or quality of that data. One crew member indicated the value added depends on the information that was being conveyed: “crew feedback is good for SA, background noise is not.” Naturally, listening to static, or non-relevant radio chatter might produce little or no SA, but to the electronic warfare officer, background listening through the radar warning receiver was a significant exception to that rule.

In reporting SA scores, some participants provided negative SA scores, which we hypothesize represents an opportunity cost that most likely trades off tactical and strategic SA. Performing task A (tactical task) consumes attentional resources that may have been spent on a hypothetical task B (strategic task) that would have yielded more SA. Thus, despite survey instructions, it is difficult to cognitively separate SA lost to higher workload (or to opportunity cost) from truly negative SA. In reality, a negative SA score would imply that by accomplishing a task, erroneous SA would be gained leading to an incorrect mental model.

Participant opinion appears to reflect this. One noted that the “largest loss of SA in terminal events is due to time constraints: time must be split between [being an] active participant in [the] event and other, coordination tasks.” While many participants generalized anecdotal evidence that some tasks add SA and some tasks taketh away, others lament that certain cockpit tasks, such as the complex manual computations required for determining aircraft performance data “can potentially take me out of the game almost completely.” Keyboard entry is considerably more complicated than simple switch actuation, and it also requires more attention devoted to ensuring that the keyboard entry is performed correctly. Furthermore, improper syntax, hitting a wrong button, and other errors create more work and perhaps distract the operator from performing other SA-generating tasks.

Perhaps what is most telling about operator opinion regarding the SA/workload tradeoff is summed up by one pilot’s opinion that, “rating is a balancing act between SA gained by the task and the SA lost by time spent completing the act.” Thus, the SA value of a heads-down multifunction display (MFD) was offset by the need to take the pilot’s eyes out of the window or HUD to consume that information. Reading raw computer data (data displayed in number or tabular form and not integrated into an MFD, HUD, or other visual display) did not yield a positive or negative SA value, potentially because of this tradeoff.

5 Future Research

While this paper explores the concept of task-dependent SA, further research is required. This survey heavily samples C-130 and other tactical aircrew. A much larger survey should be conducted that incorporates more aircraft (with and without HUD) across the spectrum of flight. More mission sets can be added including tactical and non-tactical situations at both high and low altitudes. Within each mission phase, more resolution is needed. Many participants queried “what part of airdrop/airland are you talking about?” This infers that even within the particular mission phase, SA values could change. If, for example, viewing the HUD increases value from cruise to airland missions, how much does it increase from initial approach to the actual landing? Furthermore, how do the SA values of specific tasks relate to the operator’s dynamic priorities for SA?

Aircrews have many opinions about SA, but these may not neatly fit into current academic research into SA. For example, operators appear to conflate opportunity costs of performing a task that does not generate SA at the expense of another SA-generating task with tasks that actually reduce certainty through providing misleading information. Future studies should consider exactly what “negative SA” means and account for the difference between opportunity costs and incorrect mental models.

6 Conclusion

Capturing the effect of cockpit tasks on SA can allow for the refinement of predictive SA algorithms that incorporate both workload-dependent and task-dependent SA [1]. Clearly understanding how each task builds a pilot’s mental model of his environment has clear advantages for avionics engineers when designing components, as well as for cockpit designers when incorporating those components into the aircraft’s primary machine interface. This research shows that modern avionics, such as the HUD, MFD, and other sensors can provide strong positive SA, but depending on the mission phase and other factors, they may not be significantly more advantageous than analog counterparts. Furthermore, designers must not only consider the displays, but the interface required to control those displays, as well as the attention demanded by other cockpit tasks and their opportunity costs.

7 Disclaimer

The views expressed in this paper are those of the authors and do not reflect the official policy or position of the United States Air Force, the Department of Defense, or the U.S. Government.