Keywords

1 Introduction

Successful design with good accessibility to a wide range of people requires in-depth understanding of the user, from the designers’ perspective, knowledge about (1) users’ capabilities, needs, and aspirations, (2) different scenarios that people will use products, systems and services, and (3)other factors include psychological, social, economic and other considerations should be identified [1]. In the context of a rapidly-aging society, the philosophy of inclusive design is given accumulating consideration. As the population becomes older, the needs and capabilities of people become ever more diverse [2], Neerincx et al. classify users’ abilities form a product-usage perspective [3]:

  • Sensory abilities such as seeing, hearing, touch, taste, smell and balance.

  • Physical abilities such as speech, dexterity, manipulation, mobility, strength and endurance.

  • Cognitive abilities such as intellect, memory, language and literacy.

  • Allergies can also be a significant factor in some products.

The necessity of user database, which contains users’ capabilities and contextual data, is raised to help designers to understand the nature of this diversity. A person’s multiple capabilities can reflect individual’s ability to operate or interact with products and the breadth and multivariate nature of capabilities can be best captured and represented through a database that covers multiple capability domains for a representative sample of the population [2]. Such databases that include users’ multiple capabilities information are also expected to help designers in different design phases, e.g. to get conceptual inspiration, to better understand users or to explore references to the prototype.

The current sources of regulated databases that designers can reach are almost ergonomics data, especially anthropometric data. However, there are many ‘gaps’ in the ergonomics data available to designers. For instance, in a study conducted by Nickpour and Dong, the results show that ‘all interviewed designers considered the existing anthropometric data out of date and seven out of ten believed the data was irrelevant or not applicable to their specific field of design practice. The data was also reported as being unreliable, not appealing and confusing’ [4].

To plug the gaps between existing databases and designers’ requirements, studies have been conducted to collect capability-related data, but varied shortcomings emerged [2]:

  • Databases tend to focus on a specific domain of capability, and every single survey is based on a specific sample, a product interaction often covers more than one capability, thus it is unreasonable to refer to data from different samples.

  • Some studies have captured multiple capabilities, but insufficient information was unable to be applied in supporting a practical use.

  • In sampling, the sift criteria often exclude specific population groups and often cannot represent the whole population.

  • Very few databases include data of older people.

  • Most studies were collected from western countries [5].

Additionally, technologies develop rapidly, people are always adopting new products and services, so studies based on out-of-date scales or users’ habits are no longer applicable to the new context of use. Stephanidis et al. [6] state that in the context of the emerging Information Society, we are meeting a global requirement of coping with diversity in the ‘scope and nature of tasks’ in using products.

This paper describes a pilot study that collects Chinese older people’s capability data. The main methods of the survey were derived and adapted from Towards Better Design [7], which was initiated by the University of Cambridge. That study has gained good results in gathering information of the whole age range among UK population. As a pilot study, the main aim of our survey was to verify the feasibility of this foreign study in a new background (i.e. new context and a specific target respondents: older Chinese people), as well as to explore possible factors that affect the conduct of such a survey and to conclude principles or instructions for establishing an effective older people’s database for designers in the future.

2 Methods

This pilot data collection study was conducted through a comprehensive questionnaire that contains questions as well as actual testing tasks. On one aspect, respondents were asked to verbally report some information, on the other aspect, actual performance tests were employed through three different granularities of measurements [8]:

  • Component function (such as grip strength test).

  • Activity (such as reading the LCD screen on a mobile phone).

  • Task (a product interaction that integrates number of functions and activities).

The testing items of this study were derived from the technical report of Towards Better Design with adaptations after translating the report into a Chinese version to cater for the new context and to explore insights that may emerge when employing such a survey on Chinese older people. The main adaptations were: (1) the wording and expression of the statements were modified to make it more accessible to Chinese older people and to prevent ambiguous statements and questions. (2) Testing items that involve English language (letters, numbers and words) need to be listened, read or cognized by the respondents were all replaced with Chinese. (3) Some products in the tests were replaced with other products to suit Chinese older people.

The survey collects the following information from each respondent:

  1. 1.

    Age, gender and basic household information

  2. 2.

    Health condition

  3. 3.

    Vision

  4. 4.

    Hearing

  5. 5.

    Dexterity

  6. 6.

    Mobility

  7. 7.

    Reach and stretch

  8. 8.

    Cognitive function

  9. 9.

    Technology use and experience

  10. 10.

    Product use

  11. 11.

    Physiological resources

  12. 12.

    Interface style experience

  13. 13.

    Anthropometrics

  14. 14.

    Demographics

Pretests were conducted with three older persons to refine the questions and to evaluate the time of the whole process. Finally, the expected time was set 1.5–2 h. Seven interviewers received training and observed the pretests.

3 General Results

The respondents’ age range from 50 to 80. Finally, 70 sets of valid data were collected from seven different towns and cities of China. The mean age of the samples was 64 (SD = 8.9). Basic information of the respondents is shown in Table 1.

Table 1. Basic information of the respondents
Full size table

As a face-to-face survey, it was well balanced in the quantity and quality of the inquiry. When applied in China, new issues and problems emerged in a totally new context. What should be emphasized is that the expected result of this study is not just data collected, but also the assessment of the methods and items setting applied in this survey.

After the collected data were converged, all the interviewers involved in the actual survey gathered together to assess the process and content of the survey, and the defects were put forward. First, sample bias emerged in the process, i.e. specific population groups were excluded (e.g. illiteracies and people who were unable to speak mandarin or their regional dialects were too difficult for the interviewers to understand). Secondly, the options of the questions in the same nature are not unified, both in the amount and the way of presentation, which has negative impact on the respondents’ comprehension to the questions. Thirdly, wording and expression of the statements still need to be refined, and fourthly, there are some problems regarding the items settings of the survey, which will be discussed in the next session.

4 Discussion

4.1 Self-Reporting vs. Performance Test

The methods applied in this pilot study were self-reporting and performance test. The correlations or discrepancies of the results were expected to give instructions to determine a more reasonable and effective way for the corresponding items in future research. This could lead to a new structure of a user’s comprehensive data collection work in a large-scale survey, i.e. on the premise of quality assurance, a specific capability data could be obtained just through self-reporting or performance measures, or through a mixed methods, so that the cost of such a survey could be reduced. Additionally, such a comparison may also reveal some shortcomings of this study, so that some new data collection questionnaires, devices or methods could be explored.

In our study, we found some discrepancies between self-reporting and performance tests. Figure 1. depicts the stiffness, pains or other uncomfortable conditions of the respondents; they were asked to mark any problems specified on the diagram. After overlaying all the collected results, we can see that male respondents tend to report more problems on phalangeal joints and wrists, but the performance test shows that there is no significant difference between male and female respondents (right hand: Sig. = 0.886 > 0.05; left hand: Sig. = 0.585 > 0.05).

Fig. 1.
figure 1

Visualized general heath condition self-assessment (left: Male, right: Female)

Full size image

In the memory test, 25.7 % of the respondents reported that their daily lives were “not at all limited” by their memory, but 34.3 % of them had lower performance than the average in actual test.

Fors et al. summarized the advantages and disadvantages of self-reporting and performance test [9]. Self-reporting offers a more inexpensive and easily-executed means to obtain information, and it can be realized through flexible means, including face-to-face survey, telephone and email, additionally, proxy survey could be carried out if the respondent was not able to participate in some special situations. However, its shortcomings are obvious: the reliability of self-reporting is often questioned and the results are firmly correlated with social and cultural factors, and they are sensitive to the change of these factors. Milanović et al. suggested that the indirect estimation of old peoples’ physical ability levels via questionnaire rather than actual measurement tend to lower the study’s reliability [10].

Compared with self-assessed measures, standardized performance tests have clear face validity for the task being performed, better reproducibility and are less sensitive to cognitive impairments and sociocultural factors [9]. Nevertheless, results from our study imply that a respondent may perform a test very well, but he or she does face some actual difficulties in daily life, at the same time, the environment can also affect respondent’s performance.

Related studies indicate that considerable discrepancies exist between self-reported limitations in function in independent activities of daily living and actual physical impairments. For instance, persons characterized by current joint pain or stiffness, using prescription medications, staying in urban dwelling, in depression, being female, lacking memory, having arthritis, and lack of exercise, were more likely to report higher levels of disability [11].

The reasons that lead to such discrepancies are complicated; significant portions of variance in self-reported disability can be explained by demographic, cultural, social and psychological variables such as gender, familiarity with testing activities, depression and helplessness [12–14].

It should be noticed that, form a designer’s perspective, the differences existing in users’ self-assessment and actual performance may offer some valuable insights for them to better understand the users or find a ‘design chance’. These ‘gaps’ can also be ‘opportunities’ for both researchers and designers.

As a result of the limitation of the item setting, we do not acquire enough data to compare the correlations and discrepancies of every specific capability, but the principles of related research are clarified to some extent: some categories of users’ capabilities can be obtained just through self-reporting, at the same time, capabilities that are correlated with users’ sociocultural factors should be collected through integrating self-reporting and performance measurements to achieve the reliability and to offer insights to designers.

4.2 The Selection of Products

Users’ capability of operating and interacting with products can provide direct reference for designers, in this pilot study, product-operating related testing items were both involved in questionnaire and actual performance tests.

One fundamental issue is what kind of products should be adopted in such a survey. Older people may be exposed to many specific products in their daily life. Chen et al. summarize comparatively intact product categories that are involved in the elderly’s daily living, based on different technologies, the products are classified into five categories [15]:

  • Housing and daily living, with the functions of convenience, safety, security, comfort and entertainment.

  • Communication technology, with the functions of communication and transfer information.

  • Mobility and transport, with the function of compensating or preventing physical limitations in personal mobility.

  • Health technology, with the function of home healthcare and medical assistance.

  • Education and recreation, with the functions of education and entertainment.

It is undoubtedly unrealistic to bring a detailed range of products that contains all the specific devices. But it may provide a clear structure and practical instructions to select representative products for a capability survey.

Two main categories of products were employed in our pilot study: (1) common housing products (chairs, bottles, keys, etc.), and (2) technology products (i.e. a smart phone with touch screen). The implications can be discussed from two aspects.

First, the selected products should ensure enough angles, the new trend and development of related technologies and their application on products are necessary to be taken into consideration. For instance, mobile phones are very commonly used among older people, in our study, 95.7 % of the respondents own at least one mobile phone, and 64.3 % of them use it “frequently”. Previous studies on older people’s use of mobile phone mainly based on a feature phone (normally with physical keypad and cannot download and run applications) rather than a smart phone. It is reported that China has been the biggest country smart phone assumption, and for the elderly, they are also in the context of the transition from feature phones to smart phones, especially for the ‘future’ older people, that means difficulties with smart phones that younger adults encounter now may be encountered by older adults in the future when using smart phones [16].

This big change profoundly affects the selection of products in such research; naturally, it may directly determine the detailed testing items of the survey. For example, existing research based on feature phones shows that older adults were more likely to get lost in the hierarchical menu of mobile phones than younger adults [17], however, the mainstream of smart phones now all adopt a flatting menu structure, in our survey, we have found that only 15.7 % of the respondents failed to executed the Interface Style Experience task (i.e. finding a specified menu icon).

Zhou et al. described the effects of this transition through three aspects: (1) what used to be important in feature phones (e.g., font size, icon size) is no longer so important in smart phones; (2) what used to be not so important in feature phones (e.g., connectivity) becomes important in smart phones; (3) what used to be important is still important, but it is in different embodiments (e.g., ease to find the desire functions, ease to get help, soft keys & multi-tap) [16].

Secondly, the gaps of products using between different subgroups of the elderly are reducing but still exists. We found that gender and rural-urban differences, additionally, the North and South differences, which are also addressed in most demographic-related research in China, do not have distinct effects on the purchasing, using and perceived difficulties of common housing and daily living products, but there are significant differences in communication technology products.

For instance, we raised a hypothesis that, because older people in urban and rural areas have different living habits and may conduct difference daily activities, they may show differences in product use. The statistical analysis results show that there is no significant difference (Sig.  > 0.05) between these two groups in the using frequency of ‘calling on a mobile phone’ and ‘operating TV remote control’, but in ‘taking photos’ and ‘sending text message’ we can observe significant differences (Sig. < 0.05). The corresponding perceived difficulties of the products, however, had no significant differences (Table 2). The reasons behind this phenomenon are not clear, we believe that it could be partly explained by the difference of education situation: in our research, the urban respondents’ educational levels are generally higher than the rural ones. This may imply that a standardized and unified setting of housing and daily living products in a capability survey is acceptable and reliable, while products that correlated to respondents’ cognitive capabilities should be chosen carefully.

Table 2. Significant difference between urban and rural in frequency of use and perceived difficulties in product use.
Full size table

4.3 Ceiling Effects

The elderly’s capabilities present great diversity, specifically, an old person’s performance are affected by his or her health condition, background, experience, etc. Research has shown that the results of a physical function measurement study is limited by a ceiling effect, i.e., the majority of the sample, especially the younger groups, neither report nor exhibit disabilities. This, in turn, makes it difficult to attain statistical significance in the analyses [9].

In our study, ceiling effects obviously affect the test of technology products. More than half of the respondents have never or rarely used the products specified, and it results in a great data missing in the following perceived difficulty tests (Table 3).

Table 3. The Proportion of respondents who never or rarely use the specified products
Full size table

To reduce the ceiling effects, older peoples’ characteristics and preferences should be addressed to adapt the current items and scales. For instance, Jones et al. find that older adults are more likely to seek health information, make purchases, and obtain religious information, but less likely to watch videos, download music, play games, and read blogs online [18]. That means applying some so activities or products that are common to general users in a survey focusing on the elderly is unreasonable.

5 Conclusions

Based on the results of this pilot study, some significant influencing factors in a multiple capability data collection survey are raised. First, there exists discrepancies between users’ report and their actual capabilities; capabilities that correlated with users’ sociocultural factors should be collected through integrating self-reporting and performance test to get the reliability and offer insights to designers. Secondly, products involved in this research should ensure multi angles, and address the gap of products using between different subgroups of older people. Thirdly, to reduce the ceiling effects, older peoples’ characteristics and preferences should be addressed in the survey.