Keywords

1 Introduction

For a smart watch, a smart system installed in a watch can piggyback on a smart mobile phone system and connect to internet to realize multi-functions. It can synchronize calls, messages, e-mails, photos and music etc. in a mobile phone. And data shows that the year of 2013 is a smart watch year, because technology giants such as Apple, Samsung and Google etc. all released smart watches in 2013. These two years the relation between smart watches and users are closer and closer so that users can enjoy the convenience of intelligent era more.

The screen of a smart watch is limited by size. Therefore the interface must be displayed directly and effectively. Information should be arrayed effectively. And interface should display contents to users by reasonable interface layout. Favorable interface layout can lead the visual behavior of users therefore it is important to improve the interface of a smart watch. Apple Company overcomes the limit that screen of a smart watch is too small and sets a digital wheel. Pictures are zoomed or moved by rotating the digital wheel. Apple Company also designed the interface newly. Users can see APP list on a dial plate, which makes users have a whole new experience. Good user experience embodies in good man-machine interaction, and most information of human is obtained from their own eyes, thus studying on the performance of visual search for information can make interface interaction more fast and convenient.

An eye tracker is a high technology instrument which records the eye movement track characteristics when a man is managing visual information. Eye trackers are widely used in the study of visual perception, reading and etc. When people are attracted by stimulation objects or search a target object, people’s eyes will stare at received information a little while, and people will make response, make decision and deal with it. When sight line stays at a fixed point temporarily, it is called visual focus or fixation point. The period during gazing is called fixation time. The image is formed at the center of retina when gazing.

When eyes stare or run down at something, it is a particularly important part for visual attention. Fixation time, fixation location, eye track and other eye movement process can be used as evidence basis to judge if a product is noticed. An eye tracker is used as an auxiliary mean of interface design, committing to making users feel more smooth and comfortable and producing largest vision and psychological effects on users.

2 Evaluation Method

Interface design is an important key point to decide whether users will be successful in operation and study. If the design is not appropriate, it will result in a series of problems such as users’ slow operation and learning frustration and so the use willing of users will be reduced. Eye movement track technology uses eye movement of users as standard and it can improve the operation mode of interface and enhance the use willing of users.

During usability evaluation test for interface of a smart watch, eye movement of testers are recorded by the eye tracker, the eye movement data is introduced into usability evaluation index system, intuitive and graphic eye movement data and eye movement video are evidences for analyzing interface of the smart watch and user strategies. This evaluation method is called usability evaluation method of a smart watch based on eye movement track. Simply, this study defines representation meaning of interface element and search index of interface as fixation time and fixation point, and analysis evidence is defined as fixation time hot spot diagram and eye movement video.

2.1 Fixation Time

Fixation is a relative static state of eyes. Long time fixing represents interest or confusion. Fixation time means the duration time of observing visual stimulation objects and keeping visual focus.

In this study fixation time is used to test users spend how much fixation time during the process from the operation beginning to typical task completion. Longer fixation time means that the representation meaning is worse. After experimentation, questionnaire survey is conducted to determine the implication of users’ fixation.

2.2 Fixation Point Number

Each staring means a fixation point. In this study fixation point number means total number of fixation points when users view interfaces during tasks. The number of fixation points represents cognitive process number of human for graphic interface. If the data number is big, it means testers’ absolute attention for this region or misunderstanding at a certain degree. Larger number of fixation points represents a low performance of search; and it shows that perhaps there is some problem existing in the smart watch interface.

2.3 Fixation Time Hot Spot Distribution Diagram

The hot spot diagram shows which region spends more time of the testers. In the fixation time hot spot distribution diagram more bright color represents more fixation time. It is drawn by counting sight line movement data of many testers.

2.4 Scanned Path Diagram

Space distribution of a series of fixation points and twitching of the eyelid is called scanned path diagram. Scanned path diagram provides a snapshot of tested users’ attention. But if in the test scan path is recorded long time, image will turn to be in disorder. If it is needed to record sight line movement long time, adopting hot spot diagram is appropriate.

2.5 Movement Video

Movement video can reappear sight line movement process and operation path of users during using process of smart watch. An experimenter can use eye movement data to analyze thinking process and operation behavior of users.

3 Experimental Method

3.1 Testing Users and Testing Products

There are totally 10 tested users in this experiment. They are undergraduate and graduate student from Shanghai universities. Among them 10 students are female and other students are male. Participants’ ages are between 22 and 32. Tested smart watch is Moto360. The positioning of this type of watch is an auxiliary alarm of mobile phone. Dial plate interface of Moto360 is very beautiful. It can satisfy the requirement of social communication, can also be a good accompany of sports and involves various aspects of life. Among the tested users, two used Moto360 and 6 of them have never used it.

3.2 Experimental Device

Experimental device is head-mounted eye tracker produced by German SensoMotoric Instruments Company. Its sampling frequency is 50 Hz. It mainly includes a light weight helmet and a testing computer. A camera device for photographing eyes and a camera device for photographing field are mounted on the light weight helmet. The camera device for photographing eyes is infrared camera used to photograph infrared pictures of testers’ ocular pulse. The camera device for photographing field is mainly used to photograph field images. The testing computer is mainly used to system control and record and process of data. Experimental materials are interfaces of smart watch Moto360. Experiment Center and BeGaze softwares are used to record and analyze experimental data.

3.3 Experimental Task Design

Experimental tasks of tested users are operating specified typical tasks of the smart watch. Detailed typical tasks seen in Table 1.

Table 1. Typical tasks
Full size table

3.4 Evaluation Index

Fixation time means the fixation time of all interfaces during the process from operation beginning to typical tasks operation completion for tested users. The fixation time of failure task operation for tested users is not within statistic range. Typical failure task operation includes two aspects: 1 If tested users don’t finish the typical task operation within 2 min, then typical task operation is failure. 2 If tested users don’t complete typical task operation as required, then typical task operation is failure.

Fixation point number: fixation point number of all interfaces during the process from operation beginning to typical task completion for tested users. The fixation point number of failure task operation for tested users is not within statistic range.

Task completion ratio: for each operation task, the ratio of successful typical task operation for tested users.

3.5 Experimental Procedure

Experiment Preparation and Introduction.

At the stage of experimental preparation, the experimenter checks if experimental materials, questionnaires, experimental equipments and experiment environment are all prepared. The experimenter tells the tested users the main procedure of the experiment, answer the questions proposed by tested users, and let the tested users fill in registration forms.

Eye Tracker Calibration and Eyeball Correction.

Before beginning the experiment, explain procedures and aims of the experiment to testers first, then please testers adjust position and height of a seat and sitting posture. Sight distance of testers (from eyes to screen) is kept at 60 cm. And the correction for eyeballs of testers is conducted. First determine how system detects eyeballs. Adjust the position, focus, brightness and contrast ratio of photographer to obtain optimal experimental effect (Fig. 1).

Fig. 1.
figure 1

Eye correction for the tester

Full size image

After position of head and eyeball is adjusted to an appropriate position, 5 points eye movement position calibration procedure is conducted. A cross mark will appear at the middle, upper right, lower right, upper left and lower left positions in order. Testers stare at the mark stably one by one. If there is no obvious mistake, the eye tracker will count the correspond function of screen and eyeball movement during 5 points calibration and directly convert the movement amount of eyeballs to displacement of screen coordinate (Fig. 2).

Fig. 2.
figure 2

Five point eye movement position correction

Full size image

Validation affirmation procedure is to affirm difference of the converted cross point position and the real cross point position. If the difference is within the tolerable error range that was presupposed and then experiment can be started.

Test Period.

The testers prepare for the test after understanding the experiment. Experimenters let testers know well of Moto360 smart watch interface within specified time to meet experiment requirements. Testers start test according to test instruction; testers turn on record switch. Test users start to operate the first typical task. Testers finish each task in order; and experimenters close the record switch and save (Fig. 3).

Fig. 3.
figure 3

The tester is conducting usability test experiment of Moto360 smart watch interface

Full size image

Interview After Test.

Experimenters communicate the problems, inconvenience or wrong operation users encountered during the test with testers, so that problems are found and solved.

4 Analysis on Experiment Result

4.1 Comparison of Evaluation Index

In this experiment, data process is conducted by using dedicated software of eye tracker to obtain the data of fixation time, fixation point number, average fixation time and hot spot diagram.

The following diagrams are statistic result of fixation time and fixation point number for typical tasks of smart watch interface. Compare test results combining with eye movement video and find that there is significant difference between testers in using Moto360 smart watch to finish tasks. When finishing Task3 and Task6, the average time and average fixation point number of testers using Moto360 are no different than test data of users with experienced operation, testers finish the tasks smoothly. It shows that in Moto360 smart watch interface, the representation meaning of interface element related to these two tasks is superior to others (Figs. 4 and 5).

Fig. 4.
figure 4

Average fixation time of smart watch interface

Full size image
Fig. 5.
figure 5

Average fixation point number of smart watch interface

Full size image

The table shows the task completion rate of Moto360 interface. It is seen from the table that completion rate of Task4 and Task7 is less than 100 % and the completion rate of Task7 is only 20 %, which shows that there is serious usability problem in the interface design of this task (Table. 2).

Table 2. Task completion rate of smart watch Moto360 interface
Full size table

4.2 Analyze Usability Problem and Causes of Interface Using Eye Movement Data

There is a bigger difference in fixation time, fixation point number and task completion rate evaluation index for finishing 7 tasks of Moto360 from above 3 dimensions of evaluation indexes. It is divided into 4 categories: the first category is that tasks are completed normally, the second category is that there are some obstacles during completion but relatively smooth, the third category is that there are some difficulties in completion process, and the fourth category is that completing task is very difficult. Above tasks are analyzed by combining with fixation time hot spot distribution diagram and eye movement video of tested users. It is found that big difference of performance in completing tasks is caused by different interface elements (Table. 3).

Table 3. Task completion category of smart watch Moto360 interface
Full size table

Average fixation time of looking for countdown icon is 13317.64/ms, and average fixation point number is 39.6 each one. It is found by observing eye movement video that testers regarded an alarm clock icon as a countdown icon mistakenly. Testers’eyes stared at it temporarily because the alarm clock picture contains the meaning of time, which is known by deep understand and questionnaire after test. When testers saw the countdown icon the first time, eyes paid attention to this icon and thought it temporarily after fingers slid from down to up; then associated sand clock to countdown, slid down quickly and went back to the countdown icon, at last clicked the icon to enter countdown operation interface and completed the task of looking for countdown icon. It is seen that this icon can guide the testers to set countdown function (Fig. 6).

Fig. 6.
figure 6

Find countdown icon

Full size image

Task2 is to set a countdown alarm of 1 min and 3 s, testers need to slide screen left and right to know well of setting dial plate interface of hour, minute and second. Because of the limitation of dial plate screen, one time unit can be set once; hour, minute and second can not be seen at the same time, left and right arrow need to be added for users to provide guidance for operation.

Task4 is to look for a pedometer icon, the average fixation time is 9933.257/ms, average fixation point number is 40.71429 each one, and task completion ratio is 90 %. It can be found by path scan diagram and hot spot analysis diagram that testers hesitate between a heart shape picture and a ladder shape icon. Individual testers regarded the heart rate measuring icon as the pedometer icon mistakenly thus clicking the heart rate measuring icon mistakenly. When testers saw pedometer the first time, their eyes stared at it temporarily, and most of testers clicked the pedometer icon after observing it repeatedly. Later it was understood by questionnaire and deep study that testers hesitated between heart picture and ladder icon and so it was difficult to find the pedometer icon (Fig. 7).

Fig. 7.
figure 7

Path scan diagram and hot spot analysis diagram

Full size image

Task5 is to look for the icon of setting, the average fixation time is 12744.16/ms and average fixation point number is 39.25 each one. During the process of looking for the icon of setting, wrong operation is caused by brighten and setting icons are similar, fingers slide from right to left in the middle of screen habitually, theater mode is started and so wrong operation is caused. If testers’ fingers slide at the bottom of screen, then operating icons in the middle can be avoided. It gives an inspiration to designers that visual center of the smart watch is in the middle of the dial plate. Important icons and characters should be designed according to this rule (Fig. 8).

Fig. 8.
figure 8

Causing wrong operation

Full size image

Task7 is to look for the raising wrist icon and screen brightening icon and task completion ratio is 20 %. 2 of 10 testers completed the tasks very difficultly. This icon lies in the setting. When environmental mode is “on” started, you raise a wrist, eyes stare at Moto360, at this time, screen lights up, and the screen gets dark gradually after a few seconds, but keeps lighting up all the time, the screen turned off after a time. Testers do wrong operation when they see Wrist gestures icon; Wrist gestures icon is clicked to turn pages by the action of overturning wrist when another hand is busy. It is understood by combining questionnaires after test that testers regarded the implication of a double-headed arrow as raising a wrist wrongly and testers can’t understand the implication of eyes so they don’t understand the function of this icon (Fig. 9).

Fig. 9.
figure 9

Causing wrong operation

Full size image

5 Conclusion

In this study the user test was conducted for smart watch interface. Eye movement track technology is introduced to usability evaluation of smart watch interface to measure usability level of smart watch interface. It is found from task test results that eye movement data can well evaluate smart watch interface icons and can reveal how users search their target items and information on smart watch interfaces. In this study usability evaluation method of a smart watch based on eye movement track is proposed and eye movement data is introduced to usability evaluation index system, and they are significant for guiding the usability evaluation practice of smart watch, further perfecting usability evaluation index system of smart watch and providing objective evidence for improving smart watch interface and improving usability level. The following conclusions are obtained from this study: 1 Eye movement data such as fixation time and fixation point are used as quantitative index for evaluating representation meaning of interface information structure and interface elements. 2 Eye movement data is shown by fixation time hot spot diagram which can reflect the attention distribution situation of users on interface intuitively and provide objective evidence for experimenters to analyze interface problems. 3 Eye movement video can reproduce the sight line movement situation when testers use smart watch interface. A lot of interface problems at deep levels can be found using the analysis methods of combining eye movement video and graphic eye movement data. 4 The interface usability evaluation method based on eye movement track is applied to early and middle stage of smart watch design, and the interface usability evaluation method provides objective evidence for designers to understand users’ thought and improve interface design.