Abstract
The emergence of Chatbots has attracted many firms to sell their merchandise via chats and bots. Although Chatbots have received tremendous interest, little is understood about how different usage contexts affect Chatbots’ effectiveness in mobile commerce. Due to differences in their nature, not all shopping contexts are suitable for Chatbots. To address this research gap, this study examines how contextual factors (i.e., intrinsic task complexity that embraces shopping task attributes and group shopping environment, and extrinsic task complexity that entails information intensity) affect user perceptions and adoption intentions of Chatbots as recommendation agents in mobile commerce. Applying the lenses of cognitive load theory (CLT) and common ground theory (CGT), we perform an experiment and apply quantitative analytical approaches. The results show that Chatbots are more suitable in the context of one-attribute, information-light, and group-buying tasks, whereas traditional Apps are suitable for multi-attribute, information-intensive, and single-buying scenarios. These findings make important theoretical contributions to the IT adoption literature as well as to CLT and CGT theory by contextualizing the evolving state of Chatbot commerce and providing guidelines for designing better Chatbot user experiences, thereby enhancing user perceptions and adoption intentions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We examined the literature in different domains including marketing, information systems, computer-human interactions, etc. We found that existing studies compared Chatbots and Apps using different terms for different characteristics without a united framework. Thus, during our pilot study, we asked participants to name five features of Chatbots that they think can outperform Apps. We then benchmarked these features against the extant knowledge and the Chatbot shopping examples.
MANOVA was also used to examine the effect and yielded consistent results.
References
Adipat, B., Zhang, D., & Zhou, L. (2011). The effects of tree-view based presentation adaptation on mobile web browsing. MIS Quarterly, 35(1), 99–121. https://doi.org/10.2307/23043491
Al-Maskari, A., & Sanderson, M. (2011). The effect of user characteristics on search effectiveness in information retrieval. Information Processing & Management, 47(5), 719–729. https://doi.org/10.1016/j.ipm.2011.03.002
Al-Natour, S., Benbasat, I., & Cenfetelli, R. T. (2006). The role of design characteristics in shaping perceptions of similarity: The case of online shopping assistants. Journal of the Association for Information Systems, 7(12), 821–861. https://doi.org/10.17705/1jais.00110
Al-Natour, S., Benbasat, I., & Cenfetelli, R. T. (2011). The adoption of online shopping assistants: Perceived similarity as an antecedent to evaluative beliefs. Journal of the Association for Information Systems, 12(5), 347–374. https://doi.org/10.17705/1jais.00267
Araujo, T. (2018). Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior, 85, 183–189. https://doi.org/10.1016/j.chb.2018.03.051
Arora, K. (2019). Will chatbots replace more apps in 2019? Oodles Technologies. https://www.oodlestechnologies.com/blogs/will-chatbots-replace-more-apps-in-2019/
Baddeley, A. (1992). Working memory: The interface between memory and cognition. Journal of Cognitive Neuroscience, 4(3), 281–288. https://doi.org/10.1162/jocn.1992.4.3.281
Beach, L. R., & Mitchell, T. R. (1978). A contingency model for the selection of decision strategies. Academy of Management Review, 3(3), 439–449. https://doi.org/10.2307/257535
Bickmore, T., Utami, D., Matsuyama, R., & Paasche-Orlow, M. K. (2016). Improving access to online health information with conversational agents: A randomized controlled experiment. Journal of Medical Internet Research, 18(1), 1–12. https://doi.org/10.2196/jmir.5239
Blazevic, V., & Sidaoui, K. (2022). The TRISEC framework for optimizing conversational agent design across search, experience and credence service contexts. Journal of Service Management, 33(4/5), 733–746. https://doi.org/10.1108/JOSM-10-2021-0402
Borau, S., Otterbring, T., Laporte, S., & FossoWamba, S. (2021). The most human bot: Female gendering increases humanness perceptions of bots and acceptance of AI. Psychology & Marketing, 38(7), 1052–1068. https://doi.org/10.1002/mar.21480
Brooke, C. (2017). Will chatbots end mobile apps? And why? Business 2 Community. https://www.business2community.com/mobile-apps/will-chatbots-end-mobile-apps-01853484
Campbell, D. J. (1988). Task complexity: A review and analysis. Academy of Management Review, 13(1), 40–52. https://doi.org/10.2307/258353
Chai, J., Lin, J., Zadrozny, W., Ye, Y., Stys-Budzikowska, M., Horvath, V., Kambhatla, N., & Wolf, C. (2001). The role of a natural language conversational interface in online sales: A case study. International Journal of Speech Technology, 4(3–4), 285–295. https://doi.org/10.1023/A:1011316909641
Cheng, X., Bao, Y., Zarifis, A., Gong, W., & Mou, J. (2022a). Exploring consumers’ response to text-based chatbots in e-commerce: The moderating role of task complexity and chatbot disclosure. Internet Research, 32(2), 496–517. https://doi.org/10.1108/INTR-08-2020-0460
Cheng, X., Zhang, X., Yang, B., & Fu, Y. (2022b). An investigation on trust in AI-enabled collaboration: Application of AI-driven chatbot in accommodation-based sharing economy. Electronic Commerce Research and Applications, 54, 101164. https://doi.org/10.1016/j.elerap.2022.101164
Chung, M., Ko, E., Joung, H., & Kim, S. J. (2018). Chatbot e-service and customer satisfaction regarding luxury brands. Journal of Business Research. https://doi.org/10.1016/j.jbusres.2018.10.004
Ciechanowski, L., Przegalinska, A., Magnuski, M., & Gloor, P. (2019). In the shades of the uncanny valley: An experimental study of human–chatbot interaction. Future Generation Computer Systems, 92, 539–548. https://doi.org/10.1016/j.future.2018.01.055
Clark, H. H. (1996). Using language. Cambridge University Press. https://doi.org/10.1017/CBO9780511620539
Clark, H., & Brennan, S. (1991). Grounding in communication, 127–149. In L. Resnick, B. Levine, M. John, S. D. Teasley (Eds.), Perspectives on socially shared cognition (pp. 259–292). Washington, D.C.: American Psychological Association.
Crolic, C., Thomaz, F., Hadi, R., & Stephen, A. T. (2022). Blame the bot: Anthropomorphism and anger in customer-chatbot interactions. Journal of Marketing, 86(1), 132–148. https://doi.org/10.1177/00222429211045687
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008
Elimeliah, C. (2016). Why chatbots are replacing apps? VentureBeat. https://venturebeat.com/business/why-chatbots-are-replacing-apps/
Fornell, C., & Larcker, D. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
Gadiyar, A. R. (2017). The chatbot imperative: Intelligence, personalization and utilitarian design. Retrieved from https://www.cognizant.com/whitepapers/the‐Chatbot‐imperative‐intelligence‐personalization‐and‐utilitarian‐design‐codex2469.pdf
Gefen, D., & Straub, D. W. (2003). Managing user trust in B2C e-services. e-Service Journal, 2(2), 7–24. https://doi.org/10.2979/esj.2003.2.2.7
Grover, D. (2016). Bots won’t replace apps. Better apps will replace apps. http://dangrover.com/blog/2016/04/20/bots-wont-replace-apps.html
Hair, J. F., Jr., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Multivariate data analysis. Harlow, Essex: Pearson Education Limited.
Han, M. C. (2021). The impact of anthropomorphism on consumers’ purchase decision in chatbot commerce. Journal of Internet Commerce, 20(1), 46–65. https://doi.org/10.1080/15332861.2020.1863022
Hill, J., Ford, W. R., & Farreras, I. G. (2015). Real conversations with artificial intelligence: A comparison between human–human online conversations and human–Chatbot conversations. Computers in Human Behavior, 49, 245–250. https://doi.org/10.1016/j.chb.2015.02.026
Hsu, P.-F., Nguyen, T. K., & Huang, J. (2021). Value co-creation and co-destruction in self-service technology: A customer’s perspective. Electronic Commerce Research and Applications, 46, 101029. https://doi.org/10.1016/j.elerap.2021.101029
Io, H. N., & Lee, C. B. (2017). Chatbots and conversational agents: A bibliometric analysis (pp. 215–219). Singapore: IEEE International Conference on Industrial Engineering and Engineering Management. https://doi.org/10.1109/IEEM.2017.8289883
Jenkins, M. C., Churchill, R., Cox, S., & Smith, D. (2007). Analysis of user interaction with service oriented chatbot systems. In J. A. Jacko (Ed.), Human-Computer Interaction, HCI Intelligent Multimodal Interaction Environments (pp. 76–83). https://doi.org/10.1007/978-3-540-73110-8_9
Kasilingam, D. L. (2020). Understanding the attitude and intention to use smartphone chatbots for shopping. Technology in Society, 62, 101280. https://doi.org/10.1016/j.techsoc.2020.101280
Kazmi, R. (2021). Will bots really replace apps? Koombea. https://www.koombea.com/blog/will-bots-really-replace-apps/
Kraut, R. E., Fussell, S. R., & Siegel, J. (2003). Visual information as a conversational resource in collaborative physical tasks. Journal Human-Computer Interaction, 18(1), 13–49. https://doi.org/10.1207/S15327051HCI1812_2
Kumar, V., Dixit, A., Javalgi, R. G., & Dass, M. (2016). Research framework, strategies, and applications of intelligent agent technologies (IATs) in marketing. Journal of the Academy of Marketing Science, 44(1), 24–45. https://doi.org/10.1007/s11747-015-0426-9
Lebeuf, C., Storey, M., & Zagalsky, A. (2018). Software bots. IEEE Software, 35(1), 18–23. https://doi.org/10.1109/MS.2017.4541027
Lee, Y., & Benbasat, I. (2011). Research note: The influence of trade-off difficulty caused by preference elicitation methods on user acceptance of recommendation agents across loss and gain conditions. Information Systems Research, 22(4), 867–884. https://doi.org/10.1287/isre.1100.0334
Li, L., Lee, K. Y., Emokpae, E., & Yang, S.-B. (2021). What makes you continuously use chatbot services? Evidence from Chinese online travel agencies. Electronic Markets, 31(3), 575–599. https://doi.org/10.1007/s12525-020-00454-z
Lim, Y., Lim, J., & Cho, N. (2020). An experimental comparison of the usability of rule-based and natural language processing-based chatbots. Asia Pacific Journal of Information Systems, 30(4), 832–846. https://www.earticle.net/Article/A387914
Luger, E., & Sellen, A. (2016). Like having a really bad PA: The gulf between user expectation and experience of conversational agents. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 5286–5297. https://doi.org/10.1145/2858036.2858288
Magalhães, R. (2022). Conversational user interface: The ultimate breakdown. Landbot. https://landbot.io/blog/conversational-interfaces-explained
Mahmood, T., & Ricci, F. (2009). Improving recommender systems with adaptive conversational strategies. Proceedings of the 20th ACM conference on Hypertext and hypermedia (pp. 73–82). https://doi.org/10.1145/1557914.1557930
McTear, M., Callejas, Z., & Griol, D. (2016). The conversational interface: Talking to smart devices. Springer International Publishing Switzerland. https://doi.org/10.1007/978-3-319-32967-3
Medina, G. G. (2020). From graphical interface to the conversational interface. UX Collective. https://uxdesign.cc/from-the-graphical-interface-to-the-conversational-interface-4d7ffbd05be1
Mittal, A., Agrawal, A., Chouksey, A., Shriwas, R., & Agrawal, S. (2016). A comparative study of chatbots and humans. International Journal of Advanced Research in Computer and Communication Engineering, 5(3), 1055–1057. https://doi.org/10.17148/IJARCCE.2016.53253
Moriuchi, E. (2019). Okay, Google!: An empirical study on voice assistants on consumer engagement and loyalty. Psychology & Marketing, 36(5), 489–501. https://doi.org/10.1002/mar.21192
Moriuchi, E., Landers, V. M., Colton, D., & Hair, N. (2021). Engagement with chatbots versus augmented reality interactive technology in e-commerce. Journal of Strategic Marketing, 29(5), 375–389. https://doi.org/10.1080/0965254X.2020.1740766
Naude, D. (2017). Chatbots won’t replace people. They’ll replace apps. https://medium.com/dawidnaude/chatbots-wont-replace-people-they-ll-replace-apps-8b13c555d67b
Ngai, E. W., Lee, M. C., Luo, M., Chan, P. S., & Liang, T. (2021). An intelligent knowledge-based chatbot for customer service. Electronic Commerce Research and Applications, 50, 101098. https://doi.org/10.1016/j.elerap.2021.101098
Nguyen, T. K., & Hsu, P.-F. (2022). More personalized, more useful? Reinvestigating recommendation mechanisms in e-commerce. International Journal of Electronic Commerce, 26(1), 90–122. https://doi.org/10.1080/10864415.2021.2010006
Nguyen, Q. N., Sidorova, A., & Torres, R. (2022). User interactions with chatbot interfaces vs. menu-based interfaces: An empirical study. Computers in Human Behavior, 128, 107093. https://doi.org/10.1016/j.chb.2021.107093
Olson, G. M., & Olson, J. S. (2000). Distance matters. Human-Computer Interaction, 5(2/3), 139–178. https://doi.org/10.1207/S15327051HCI1523_4
Ortega, B. H., Martinez, J. J., & Hoyos, M. J. M. D. (2006). Analysis of the moderating effect of industry on online behavior. Online Information Review, 30(6), 681–698. https://doi.org/10.1108/14684520610716162
Pan, J. (2019). Are private group chats the future of messaging marketing? Landbot. https://landbot.io/blog/group-chats-marketing.
Payne, J. (1982). Contingent decision behavior. Psychological Bulletin, 92(2), 382–402. https://doi.org/10.1037/0033-2909.92.2.382
Pereira, M. J., Coheur, L., Fialho, P., & Ribeiro, R. (2016). Chatbots’ greetings to human‐computer communication. arxiv. https://doi.org/10.48550/arXiv.1609.06479
Pichsenmeister, D. (2017). Bot trends 2017. Medium. https://medium.com/@pichsenmeister/bot-trends-2017-78f9ecd6317f
Pounder J, (2016). Huddle — Humanity in the machine. Mindshare. https://www.mindshareworld.com/sites/default/files/mindshare_huddle_humanity_machine_2016_0.pdf
Pradana, A., Sing, G. O., & Kumar, Y. J. (2017). SamBot-intelligent conversational bot for interactive marketing with consumer-centric approach. International Journal of Computer Information Systems and Industrial Management Applications (6), pp. 265–275 (ISSN 2150–7988).
Radziwill, N. M., & Benton, M. C. (2017). Evaluating quality of chatbots and intelligent conversational agents. Software Quality Professional, arXiv. https://doi.org/10.48550/arXiv.1704.04579
Rapp, A., Curti, L., & Boldi, A. (2021). The human side of human-chatbot interaction: A systematic literature review of ten years of research on text-based chatbots. International Journal of Human-Computer Studies, 151, 102630. https://doi.org/10.1016/j.ijhcs.2021.102630
Reddy, R. (2018). Chatbots Vs apps: What does your business need? BotCore. https://botcore.ai/blog/chatbots-vs-apps-what-does-your-business-need/
Romero, N., & Markopoulos, P. (2005). Common ground as privacy mechanism in the use of awareness systems. Proceedings of the 2005 IFIP TC13 international conference on Human-Computer Interaction, pp.1006–1009. https://doi.org/10.1007/11555261_91
Rowley, J. (2000). Product searching with shopping bots. Internet Research, 10(3), 203–214. https://doi.org/10.1108/10662240010331957
Sajjad, F. (2019). Will chatbots kill apps and websites? https://cyfuture.com/blog/will-chatbots-kill-apps-and-websites/
Sands, S., Ferraro, C., Campbell, C., & Tsao, H. Y. (2020). Managing the human–chatbot divide: How service scripts influence service experience. Journal of Service Management. https://doi.org/10.1108/JOSM-06-2019-0203
Schlicht, M. (2016). How bots will completely kill websites and mobile apps? https://chatbotsmagazine.com/how-bots-will-completely-kill-websites-and-mobile-apps-656db8e6fc03
Schuetzler, R., Grimes, M., Giboney, J., & Buckman, J. (2014). Facilitating natural conversational agent interactions: lessons from a deception experiment. Auckland: Proceedings of the 35th International Conference on Information Systems.
Schultze, U., & Brooks, J. A. M. (2019). An interactional view of social presence: Making the virtual other “real.” Information Systems Journal, 29(3), 707–737. https://doi.org/10.1111/isj.12230
Speier, C., & Morris, M. G. (2003). The influence of query interface design on decision-making performance. MIS Quarterly, 27(3), 397–423. https://doi.org/10.2307/30036539
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1016/0364-0213(88)90023-7
Sweller, J. (2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22(2), 123–138. https://doi.org/10.1007/s10648-010-9128-5
Sweller, J. (2011). Cognitive load theory. In Psychology of learning and motivation, 55, pp. 37–76, Academic Press. https://doi.org/10.1016/B978-0-12-387691-1.00002-8
Venkatesh, V., Thong, J. Y. L., Chan, F. K. Y., Hu, P. J., & Brown, S. A. (2011). Extending the two-stage information systems continuance model: Incorporating UTAUT predictors and the role of context. Information Systems Journal, 21(6), 527–555. https://doi.org/10.1111/j.1365-2575.2011.00373.x
Venkatesh, V., Aloysius, J., Hoehle, H., & Burton, S. (2017). Design and evaluation of auto-id enabled shopping assistance artifacts in customers’ mobile phones: Two retail store laboratory experiments. MIS Quarterly, 41(1), 83–113. https://doi.org/10.25300/MISQ/2017/41.1.05
Wood, R. E. (1986). Task complexity: Definition of the construct. Organizational Behavior and Human Decision Processes, 37(1), 60–82. https://doi.org/10.1016/0749-5978(86)90044-0
Xu, D. J., Benbasat, I., & Cenfetelli, R. T. (2014). The nature and consequences of trade-off transparency in the context of recommendation agents. MIS Quarterly, 38(2), 379–406. https://doi.org/10.25300/MISQ/2014/38.2.03
Zamora, J. (2017). Rise of the chatbots: Finding a place for artificial intelligence in India and US. Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion, pp. 109–112.
Zhang, D. (2003). Delivery of personalized and adaptive content to mobile devices: A framework and enabling technology. Communications of the Association for Information Systems, 12, 183–202. https://doi.org/10.17705/1CAIS.01213
Zhu, L., Benbasat, I., & Jiang, Z. (2010). Let’s shop online together: An empirical investigation of collaborative online shopping support. Information Systems Research, 21(4), 872–891. https://doi.org/10.1287/isre.1080.0218
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible Editor: Samuel Fosso Wamba
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Hsu, PF., Nguyen, T.(., Wang, CY. et al. Chatbot commerce—How contextual factors affect Chatbot effectiveness. Electron Markets 33, 14 (2023). https://doi.org/10.1007/s12525-023-00629-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s12525-023-00629-4