Keywords

1 Introduction and Theory

In aviation psychology, the criterion of situation-awareness has become central for the evaluation of human-machine interfaces. What is often overlooked, however, is that usually situations are not constituted from random patterns of indicators but from events having a well-defined structure in space and time [1, 2]. Experience with these events helps domain experts to achieve a high degree of efficacy while using their mental resources, because not all aspects of a familiar situation have to be processed, but it is only necessary to identify the type of the event and then the specifics of the actual event. Experts therefore can focus on the important aspects of the event and only need to check the situations with regard to deviations from the standard. Therefore, they need to spend fewer resources on the less relevant aspects of the situation and can be more efficient in the identification of critical aspects of the event. Hence, an important precondition for the quality of situation-awareness is the quality of the representation of the events in question.

Because of their potential to present the three-dimensional airspace and the required information in a natural way, 3D visualizations possess the potential for providing high quality representations of events for the air-traffic-controller. In the following, we will use the relevant details and results of a larger experiment [3] to illustrate how 3D-visualizations facilitate the representation of event structure of potential conflict situations in air-traffic-control.

2 Methodology

12 air-traffic-controllers (ATCOs) and 12 pilots were recruited and asked to judge the outcome of 108 representative safety critical air-traffic events in each of the four visualizations described below. All events consisted of two converging aircrafts that were going to reach the closest point of approximation in 45 s after the scenario has been started. These events were created by variation of the convergence angle between them (0°, 90° or 180°) and vertical movement of none, one, or both objects. Whenever there was vertical movement, one of the two aircraft descended with 1500 ft/min. In the case of both aircraft moving vertically, one climbed with 500 ft/min. Furthermore, three different distances were implemented to create vertical and horizontal separations. While the former were created by vertically separating the point of the closest approximation by 1000 ft, 2000 ft and 3000 ft, the latter were separated by 1 NM, 2 NM and 3 NM. This way, a total of 54 different events that showed separations resulted. To create an equal number of conflicts, the separation distances of these 54 events were set to zero. Hence, the likelihood for a conflict or separation to be displayed was 50 %.

In order to simulate the increased mental workload of peak-traffic-situations, in a second run using only the 36 events with the 90° angle of convergence, subjects not only had to judge the possibility of a collision but also to attend to a secondary auditory task (aat). As in the previous run, a six-point-rating scale was shown directly after displaying each event for exactly 10 s which masked the scenario with an entry mask for judging the outcome of the event with one of the following options: certainly conflicting, probably conflicting, perhaps conflict, perhaps not conflicting, probably not conflicting, and certainly not conflicting.

After judging the events with and without the additional auditory task, the procedure was repeated with the next visualization. Both the sequence of the events and the sequence of the visualizations were randomized.

Since a negative transfer from the previous training of the ATCOs could impair the comparison between the 2D visualization they are familiar with and the novel 3D visualizations, we also recruited pilots for comparison. Due to the fact that similar ability tests are applied in the selection for both professions, ATCOs and pilots can be considered as comparable regarding their cognitive abilities. In contrast to pilots, however, air-traffic-controllers possess extensive training and experience in judging safety-critical air-traffic-events using the 2D top-view visualization. Figure 1 shows all four modes of visualizations that have been used to display the air-traffic-events.

Fig. 1.
figure 1

Modes of visualization: 2D standard display in air-traffic-control (2D), 3D stereoscopic view from above (3 Da), 3D stereoscopic side-wise view with drop-lines to the position over ground (3Ddl), 3D same as 3Ddl but without the drop-line (3D) [3].

Full size image

The 3D visualizations have been designed based on theoretical considerations concerning their specific benefits or drawbacks for air-traffic-control. The 2D visualization represented the display currently used in air-traffic-controller workstations. To create comparable conditions, all visualizations contained predictor lines attached to the aircraft symbols that indicate the flight direction and speed. Their length is linearly connected to the velocity. Furthermore, labels indicated each object’s altitude and their velocity numerically. Arrows close to the altitude representation qualitatively indicated altitude changes by pointing up- or downward. A reference grid with a width of 1.5 NM, drawn on the ground, supported the perception of lateral distances. All 3D visualizations used stereoscopy. The participants wore polarized glasses and their position was tracked by an infrared system so the view could automatically be adapted to their viewing position. With both 2D and 3 Da, the participants were provided with a vertical top view that provided a more precise perception of horizontal distances and convergence angles on the horizontal plane than the 3D side-views 3Ddl and 3D, because ambiguities along the line of sight were reduced. 3Ds additionally showed the altitude analogously by using drop-lines. 3Ddl and 3D supposedly amplified the perception of vertical distances, but line-of-sight ambiguities were expected to decrease performance with them when judging horizontal distances. In the case of 3Ddl, however, drop-lines were expected to reduce these negative effects.

3 Results

3.1 Conflict Detection Rate

Since controllers need to apply a most conservative criterion to ensure maximum safety, they will only decide not to intervene when they are absolutely certain that no conflict can occur. Therefore, all incidents not judged as perfectly safe, that is, certainly no, (one of the five categories ranging from certainly yes to probably no) can be considered as situations in which the controller would act in order to mitigate the risk of a collision. Hence, all scenarios showing a conflict and being assigned to one of these categories are considered as correctly identified conflicts. Conflicts that are classified with certainly no constitute misses. Figure 2 shows the conflict detection rates (number of correctly identified conflicts divided by the total number of displayed conflicts) achieved by the ATCOs with each mode of visualization and in both mental workload conditions with and without the additional auditory task.

Fig. 2.
figure 2

Conflict detection rates (with standard errors) of the ATCOs with and without the additional auditory task (aat) [3].

Full size image

An analysis of variance indicates the statistical significance of the difference between the visualizations when the numbers of false alarms is statistically controlled (p = 0.022; F(3, 384) = 3.244; η2 = 0.027). Though not statistically significant (p = 0.06; η2 = 0.021), the manipulation of mental workload results in a noteworthy interaction with the visualizations, indicating a higher negative impact of mental workload on performance with 2D compared to 3D. The absence of further significant effects indicates the validity of these results also for the Pilots.

Without the additional auditory task, all modes of visualization result in similar conflict detection rates. The higher mental workload, however, induces lower detection rates, especially for 2D. In the case of the 3D side-wise view with drop-lines, the additional auditory task causes the least decrease in performance.

3.2 Judgment Certainty

Further analyses reveal differences regarding judgment certainty that depend on the spatial dimension by which a situation has to be assessed. In a situation in which the aircraft moves are vertically separated allows to exclude the possibility of a conflict by assessing vertical distance information only, similarly, the identification of a horizontally separated situation is possible on the mere analysis of horizontal distance information. However, in separations, where both the horizontal and the vertical dimension have to be assessed to exclude a separation when the scenario shows a conflict, the controller has to have a three-dimensional mental model.

For the analysis of judgment certainty, a measure was created to allow reporting the judgment certainty for each dimension separately with a value between 0 and 1 by assigning specific values to the six response categories. In the case of a conflict, the value 0 represents the category certainly no, and the value 1 the category certainly yes, whereas separations receive the value 0 for the category certainly yes, and the value 1 for certainly no. The values 0.2, 0.4, 0.6, and 0.8 are assigned to the categories probably and maybe respectively. Based on these converted judgements, an analysis of variance is conducted. An analysis of variance including the factors visualization, dimension, mental workload condition as well as group membership shows a significant interaction between visualization and dimension (p = 0.025; F(6, 43) = 2.453; η2 = 0.054). Since no other noteworthy interaction effect has been found, the differences in judgment certainty between the visualizations are independent from the mental workload. This result is valid for both the ATCOs and the Pilots. To illustrate this interaction effect, Fig. 3 illustrates the judgement certainty of the ATCOs in the condition with the additional auditory task. The values that result with 2D have been adjusted to zero to serve as a baseline and facilitate the comparison.

Fig. 3.
figure 3

Judgement certainty of the ATCOs in the condition with additional auditory task (aat). The values have been adjusted to zero so that 2D serves as a baseline [3].

Full size image

While in the case of conflicts no noteworthy differences in regard to judgement certainty appear between the modes of visualization, the use of a 3D side-wise view with drop-lines leads to a higher judgement certainty than the 2D reference. However this advantage of 3D disappears without the drop-lines. In cases of horizontal separation, 2D results in a higher judgement certainty than any of the 3D visualizations.

3.3 Learning Effects

In order to investigate how the performance develops over time, the performance with which the participants were able to discriminate between conflict and separation has been analysed using the areas under the receiver operating characteristic curves (AUC). Since the AUC values are independent from the underlying judgment certainty and decision criterion, this method is especially suitable for reporting and comparing learning effects due to 2D and 3D visualizations. The reason for this lies in the fact that ATCOs are familiar with the former, and that they lack training with the latter. Therefore, a higher degree of uncertainty might result from using a novel visualization which would impair the reliability of the results. The same is applies to the comparison between ATCOs and pilots, because these two groups might apply different response criteria due to their different experience with the air-traffic-situation. The AUCs expresses the likelihood for correctly discriminating between the presented conflicts and separations. Figure 4 shows the resulting AUCs of the ATCOs’ and the Pilots’ ratings for both the 2D reference and the 3Ddl visualization. Each measurement period (MP) shows the AUC values that result from the respective third of the displayed events (48 per MP and visualization), and are independent from the cognitive workload condition, because both have been pooled for the analysis. The results are independent from the mental workload condition because both conditions have been pooled for the analysis.

Fig. 4.
figure 4

Discrimination performance (with standard errors) of the ATCOs and the Pilots per measurement period (MP) [4].

Full size image

The results show that on average the Air-Traffic-Controllers achieve a higher discrimination performance than the Pilots. Albeit initially better with the 2D visualization, the Air-Traffic-Controllers on average achieve a higher performance with the 3D side-wise view with drop-lines in the third measurement period. In contrast to these results, Pilots achieve a higher discrimination performance with the 3D side-wise view from the very beginning and improve their performance with the 2D reference over time.

4 Discussions and Conclusions

The results indicate that a stereoscopic 3D side-wise view with drop-lines represents the event structure best. It facilitates the efficacy of the controllers by increasing their conflict detection rate without compromising the efficiency by altering the likelihood of false alarms.

Adding a secondary task that requires the controllers to process two streams of information simultaneously underpins the advantage of the direct event mapping with the 3D side-wise view. This indicates benefits of 3D-representations in air-traffic-control concerning the mental workload and awareness of safety critical events.

The findings in regard to the differences in judgment certainty between the visualizations indicate where in air-traffic control 3D-representations should be implemented. Though the assessment of conflict situations can be achieved with equal certainty in all visualizations, different strengths and weaknesses of the mode of visualization become apparent in situations of ‘near misses’. Since 3D visualizations with drop-lines enhance the controller’s judgment certainty in the case of vertical separations, 3D-Displays seem to be especially beneficial when used for air-traffic-control tasks, which require vertical distances judgments. In contrast, 2D visualizations facilitate controller’s judgment certainty of horizontal distances and therefore should be used accordingly. Hence, the demands of the task determine which visualization provides the best quality of the event representation.

The analysis of the learning effects should give rise to further research on the usefulness of 3D-Displays for air-traffic-control, because air-traffic-controllers apparently require little time to make use of the benefits inherent in 3D visualizations. Furthermore, 3D-Displays might become even more beneficial for specific trainings aimed at conflict assessment; this becomes apparent from the learning process of pilots who have not undergone conflict judgment training and lack the experience of the air-traffic-controllers: from the very beginning of the testing they achieve a higher discrimination performance with 3D as compared to 2D.