Abstract
The credibility of research on information system security is challenged by inconsistent results and there is an ongoing discussion about research methodology and its effect on results within the employee non-/compliance to information security policies literature. We add to this discussion by investigating discrepancies between what we cl/aim to measure (theoretical properties of variables) and what we actually measure (respondents’ interpretations of our operationalized variables). The study asks: (1) How well do respondents’ interpretations of variables correspond to their theoretical definitions? (2) What are the characteristics and causes of any discrepancies between variable definitions and respondent interpretations? We report a pilot study including interviews with seven respondents to understand their interpretations of the variable Perceived severity from the Protection Motivation Theory (PMT).
We found that respondents’ interpretations differ substantially from the theoretical definitions which introduces error in measurement. There were not only individual differences in interpretations but also, and more importantly, systematic ones; When questions are not well specified, or do not cover respondents’ practice, respondents make interpretations based on their practice. Our results indicate three types of ambiguities, namely (i) Vagueness in part/s of the measurement item causing inconsistencies in interpretation between respondents, (ii) Envision/Interpret ‘new’ properties not related to the theory, (iii) ‘Misses the mark’ measurements whereby respondents misinterpret the fundamentals of the item. The qualitative method used proved conducive to understanding respondents’ thinking, which is a key to improving research instruments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aurigemma, S., Mattson, T.: Generally speaking, context matters: making the case for a change from universal to particular ISP research. J. Assoc. Inf. Syst. 20(12), 7 (2019)
Barlette, Y., Gundolf, K., Jaouen, A.: Toward a better understanding of SMB CEOs’ information security behavior: Insights from threat or coping appraisal. J. Intell. Stud. Bus. 5(1) (2015)
Bazeley, P.: Qualitative Data Analysis Practical Strategies, 2nd edn. Sage, London (2013)
Blythe, J.M., Coventry, L.: Costly but effective: comparing the factors that influence employee anti-malware behaviours. Comput. Hum. Behav. 87, 87–97 (2018)
Burns, A.J., Posey, C., Roberts, T.L., Lowry, P.B.: Examining the relationship of organizational insiders’ psychological capital with information security threat and coping appraisals. Comput. Hum. Behav. 68, 190–209 (2017)
Boudreau, M.C., Gefen, D., Straub, D.W.: Validation in information systems research: a state-of-the-art assessment. MIS Q. 1–16 (2001)
Cram, W.A., D’arcy, J., Proudfoot, J.G.: Seeing the forest and the trees: a meta-analysis of the antecedents to information security policy compliance. MIS Q. 43(2), 525–554 (2019)
Desimone, L.M., Le Floch, K.C.: Are we asking the right questions? Using cognitive interviews to improve surveys in education research. Educ. Eval. Policy Anal. 26(1), 1–22 (2004)
Gerdin, M., Grönlund, Å., Kolkowska, E.: Use of protection motivation theory in non-compliance research (2021)
Haag, S., Siponen, M., Liu, F.: Protection motivation theory in information systems security research: a review of the past and a road map for the future. ACM SIGMIS Database: DATABASE Adv. Inf. Syst. 52(2), 25–67 (2021)
Hooper, V., Blunt, C.: Factors influencing the information security behaviour of IT employees. Behav. Inf. Technol. 39(8), 862–874 (2020)
Ifinedo, P.: Understanding information systems security policy compliance: an integration of the theory of planned behavior and the protection motivation theory. Comput. Secur. 31(1), 83–95 (2012)
Johnston, A.C., Warkentin, M., Siponen, M.: An enhanced fear appeal rhetorical framework. MIS Q. 39(1), 113–134 (2015)
Karjalainen, M., Sarker, S., Siponen, M.: Toward a theory of information systems security behaviors of organizational employees: a dialectical process perspective. Inf. Syst. Res. 30(2), 687–704 (2019)
Karlsson, F., Karlsson, M., Åström, J.: Measuring employees’ compliance – the importance of value pluralism. Inf. Comput. Secur. 25(3), 279–299 (2017)
Li, H., Luo, X.R., Chen, Y.: Understanding information security policy violation from a situational action perspective. J. Assoc. Inf. Syst. 22(3), 7398–7772 (2021)
Luft, J., Shields, M.D.: Mapping management accounting: graphics and guidelines for theory-consistent empirical research. Acc. Organ. Soc. 28(2–3), 169–249 (2003)
Ma, X.: IS professionals’ information security behaviors in Chinese IT organizations for information security protection. Inf. Process. Manage. 59(1), 102744 (2022)
MacKenzie, S.B., Podsakoff, P.M., Podsakoff, N.P.: Variable measurement and validation procedures in MIS and behavioral research: Integrating new and existing techniques. MIS Q. 35, 293–334 (2011)
Moody, G.D., Siponen, M., Pahnila, S.: Toward a unified model of information security policy compliance. MIS Q. 42(1) (2018)
Mou, J., Cohen, J.F., Bhattacherjee, A., Kim, J.: A test of protection motivation theory in the information security literature: a meta-analytic structural equation modeling approach. J. Assoc. Inf. Syst. 23(1), 196–236 (2022)
Posey, C., Roberts, T.L., Lowry, P.B.: The impact of organizational commitment on insiders’ motivation to protect organizational information assets. J. Manag. Inf. Syst. 32(4), 179–214 (2015)
Rajab, M., Eydgahi, A.: Evaluating the explanatory power of theoretical frameworks on intention to comply with information security policies in higher education. Comput. Secur. 80, 211–223 (2019)
Rogers, R.W.: A protection motivation theory of fear appeals and attitude change1. J. Psychol. 91(1), 93–114 (1975)
Rogers, R.W.: Cognitive and physiological processes in fear-based attitude change: a revised theory of protection motivation. In: Cacioppo, J., Petty, R. (eds.) Social Psychophvsiology. A Source Book, pp. 153–176. Guilford, New York (1983)
Siponen, M., Vance, A.: Guidelines for improving the contextual relevance of field surveys: the case of information security policy violations. Eur. J. Inf. Syst. 23(3), 289–305 (2014)
Siponen, M., Mahmood, M.A., Pahnila, S.: Employees’ adherence to information security policies: an exploratory field study. Inf. Manag. 51(2), 217–224 (2014)
Sommestad, T., Karlzén, H., Hallberg, J.: The sufficiency of the theory of planned behavior for explaining information security policy compliance. Inf. Comput. Secur. 23(2), 200–217 (2015)
Sommestad, T., Hallberg, J., Lundholm, K., Bengtsson, J.: Variables influencing information security policy compliance: a systematic review of quantitative studies. Inf. Manag. Comput. Secur. 22(1), 42–75 (2014)
Straub, D.W.: Validating instruments in MIS research. MIS Q. 147–169 (1989)
Vance, A., Siponen, M., Pahnila, S.: Motivating IS security compliance: insights from habit and protection motivation theory. Inf. Manag. 49(3–4), 190–198 (2012)
Vrhovec, S., Mihelič, A.: Redefining threat appraisals of organizational insiders and exploring the moderating role of fear in cyberattack protection motivation. Comput. Secur. 106, 102309 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 IFIP International Federation for Information Processing
About this paper
Cite this paper
Gerdin, M., Grönlund, Å., Kolkowska, E. (2023). What Goes Around Comes Around; Effects of Unclear Questionnaire Items in Information Security Research. In: Furnell, S., Clarke, N. (eds) Human Aspects of Information Security and Assurance. HAISA 2023. IFIP Advances in Information and Communication Technology, vol 674. Springer, Cham. https://doi.org/10.1007/978-3-031-38530-8_37
Download citation
DOI: https://doi.org/10.1007/978-3-031-38530-8_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-38529-2
Online ISBN: 978-3-031-38530-8
eBook Packages: Computer ScienceComputer Science (R0)
