Keywords

1 Introduction

The proportion of mental work in everyday life is increasing in the advanced information society. The load and burden owing to mental work is called mental work-load (MWL) [1]. Appropriate evaluation and management of MWL is important for reducing human error and health damage [2,3,4,5].

Appropriate evaluation and management of MWL is important to reduce human error and health damage [2,3,4,5]. Because MWL affects autonomic nervous activity, estimating MWL by evaluating autonomic nervous activity has been investigated. Some studies have reported on measuring nasal skin temperature (NST) using infrared thermography and evaluating autonomic nervous activity during MWL. NST is an indicator that reflects autonomic nerve activity and the effectiveness of MWL evaluation. This method extracts the forehead and nose from the facial thermal image to determine the temperature difference using NST to evaluate and estimate autonomic nervous activity. Skin temperature is measured with less restraint and non-contact compared to other bioelectric signal indices. Therefore, it is suitable for measurement in a real work scene. Existing studies have reported that MWL may affect NST and the skin temperature of the lips [6,7,8,9]. Therefore, the analysis range was extended from the nose to the entire face. Then, the variance of the skin temperature of the entire face was used as an index of autonomic nervous activity. This can estimate and evaluate autonomic nervous activity with higher accuracy than the approaches used in existing studies. However, because the index uses a thermal image of the entire face, its effect on hair and wearing glasses is not considered. In an actual environment, it is difficult to evaluate the temperature of an entire face if the forehead is covered by hair or wearing glasses.

Therefore, this study considers the lower half of the face area; the area with less effect from glasses and hair. Temperature analysis was performed on the characteristic areas such as nose, cheeks, lips, and the entire area under the face. Additionally, we compared the temperature distribution characteristics of these areas with psychological and behavioral indices and evaluated the MWL status of the subjects.

2 Experiment

Experiments were carried out to acquire facial thermal images of different MWL states. Subjects performed a consuming MWL task.

2.1 Experimental Procedure

Figure 1 shows the experimental system is shown in Fig. 1. Thermal images were taken when after a subject solved the a mental arithmetic calculation. An infrared thermography device (ViewOhreIMAGING XA0350) was placed at a distance of 1 m horizontally from the nose of the subject. The thermal image size was 320 * 240 pixels, and the sampling period was 1 s. A PC display and numeric keypad were placed uponon the desk.

Figure 2 shows the experimental protocol is shown in Fig. 2 shows. Subjects rested for were given a 3 min rest, period in a sitting position. After this initial rest period. Subsequently, subjects began the mental arithmetic calculation task and continued this task that lasts for 10 min. After completion of the task, subjects again rested for took 3 min rest, thereby completing the experiment. The calculation involved the addition of two integers, each of which was between 10 and 99. The subject inputs the answer of the 4 calculations displayed on the PC by using the numeric keypad. The calculation was performed and displayed for 20 s. After 20 s, the following calculation will displayed on the PC, regardless of whether the subject answers the calculation within 20 s or not. The subjects were 6 healthy adults, from 22 to 28 years old, who were well had adequate rested the night before the experiment.

Fig. 1.
figure 1

Experimental system.

Full size image
Fig. 2.
figure 2

Experimental procedure.

Full size image

2.2 Analysis Area of Lower Half of Face

The analysis area is shown in Fig. 3. From Fig. 3, (a) the nose tip, (b) the right cheek, (c) the left cheek, (d) the lip, and (e) the lower half of the face, the average temperature and the variance of each area is calculated. The nose tip denotes the width of the nose, the height denotes the center of the back of the nose that did not interfere with the glasses. The width of the lip region was approximately the same as in (a), and the height was extracted as an ellipse from the chin to the nose tip. The cheek was set in the remaining area and analyzed. Additionally, for the evaluation values of the average and variance values, the average value from the start to the end of each mental arithmetic calculation task was calculated, and the tendency of the temperature transition in each trial was examined.

Fig. 3.
figure 3

Analysis area.

Full size image

2.3 Behavioral and Psychological Indicators

As behavior indicators, the average, and standard deviation of the time taken to answer, and the number of incorrect answers were calculated for each trial of the mental arithmetic calculation task. For psychological indicators, VAS was used. The VAS evaluates the subjective degree by marking it on a straight line. In this study, the evaluation was given on a scale of 100 according to the psychological score at the start of the experiment.

Five evaluation items were used: pleasure, concentration, fun, energy, and cold. For each subject, the items of each scale were converted into a standardized score T using Eq. (1), and the average scores and standard deviations were calculated for each individual by the psychological score during the task. The psychological score at the start of the experiment was set as a reference value, and the difference value from the psychological score after the task was set.

$$\begin{aligned} \mathrm{T} = \text {(VAS value} - \text {Average)SD} \end{aligned}$$
(1)

2.4 Results and Discussion

Behavioral Performance. Behavioral performance was divided into subjects whose answer time and number of errors increased and those whose answer time decreased. The former is influenced by monotonous feelings and fatigue from the task over time. The latter is influenced by habituation.

Psychological Evaluation. The results of VAS are shown in Fig. 4. Considering the psychological effect of the computational task, most subjects exhibited negative emotions such as discomfort, distraction, and fatigue over time. Some subjects changed to positive emotions such as pleasantness and concentration.

Fig. 4.
figure 4

Analysis area.

Full size image
Fig. 5.
figure 5

Average temperature (Sub. A).

Full size image
Fig. 6.
figure 6

Variance (Sub. A).

Full size image

Physiological Evaluation. The results of the subject A are shown in Figs. 5 and 6.

A temperature fluctuation caused by the calculation task in other areas is indicated in Fig. 5. However, the area with high changes in the average temperature is on the nose (a). Each time the task was performed, area (a) fell below the other areas. However, some subjects exhibited small changes, and there were individual differences.

In Fig. 6, areas (a) and (e) are shown as those with high changes in the variance of temperature. Although there were individual differences in the magnitude of the values, the tendency was similar. From above, it was suggested that the nose and lower half of the face can affect the variance of temperature by MWL. In other subjects, the variance increased with each trial.

Conclusion. This study considers the lower half of the face area that is less affected by glasses and hair. We compared the temperature distributions of these areas with psychological and behavioral indices and evaluated the MWL status of the subjects. Consequently, the effect of MWL can be estimated using the variance of the nose and lower half of the face. In further studies, we will develop a system that can perform real-time processing.