Abstract
E-commerce customers frequently seek detailed product information for purchase decisions, commonly contacting sellers directly with extended queries. This manual response requirement imposes additional costs and disrupts buyer’s shopping experience with response time fluctuations ranging from hours to days. We seek to automate buyer inquiries to sellers in a leading e-commerce store using a domain-specific federated Question Answering (QA) system. The main challenge is adapting current QA systems, designed for single questions, to address detailed customer queries. We address this with a low-latency, sequence-to-sequence approach, Message-to-Question (M2Q). It reformulates buyer messages into succinct questions by identifying and extracting the most salient information from a message. Evaluation against baselines shows that M2Q yields relative increases of 757% in question understanding, and 1,746% in answering rate from the federated QA system. Live deployment shows that automatic answering saves sellers from manually responding to millions of messages per year, and also accelerates customer purchase decisions by eliminating the need for buyers to wait for a reply.
B. Fetahu and T. Mehta—Contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Our human annotators are expert in-house annotators that provide their relevance judgements based on a pre-determined annotation protocol, which was designed specifically for this task.
- 2.
Due to privacy regulations, we cannot use external API-based LLMs like ChatGPT.
- 3.
We assess the proportion of questions answered when the QA confidence surpasses a threshold.
- 4.
If unsatisfied with an instant answer, users can forward their question to the seller.
References
Ahearne, M., Atefi, Y., Lam, S.K., Pourmasoudi, M.: The future of buyer-seller interactions: a conceptual framework and research agenda. J. Acad. Mark. Sci. 50, 22–45 (2022). https://doi.org/10.1007/s11747-021-00803-0
Cao, Y., et al.: TASA: deceiving question answering models by twin answer sentences attack. arXiv preprint arXiv:2210.15221 (2022)
Chen, M., et al.: The JDDC corpus: a large-scale multi-turn Chinese dialogue dataset for e-commerce customer service. In: Proceedings of the 12th Language Resources and Evaluation Conference, pp. 459–466 (2020)
Chiang, W.L., et al.: Vicuna: an open-source chatbot impressing GPT-4 with 90%* ChatGPT quality, March 2023. https://lmsys.org/blog/2023-03-30-vicuna/
Chung, H.W., et al.: Scaling instruction-finetuned language models. CoRR abs/2210.11416 (2022). https://doi.org/10.48550/arXiv.2210.11416
Cui, L., Huang, S., Wei, F., Tan, C., Duan, C., Zhou, M.: SuperAgent: a customer service Chatbot for e-commerce websites. In: Proceedings of ACL 2017, System Demonstrations, pp. 97–102 (2017)
Deng, Y., Zhang, W., Yu, Q., Lam, W.: Product question answering in e-commerce: a survey. arXiv preprint arXiv:2302.08092 (2023)
Do, X.L., Zou, B., Pan, L., Chen, N., Joty, S., Aw, A.: CoHS-CQG: context and history selection for conversational question generation. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 580–591 (2022)
Faustini, P., Chen, Z., Fetahu, B., Rokhlenko, O., Malmasi, S.: Answering unanswered questions through semantic reformulations in spoken QA. In: Sitaram, S., Klebanov, B.B., Williams, J.D. (eds.) Proceedings of The 61st Annual Meeting of the Association for Computational Linguistics: Industry Track, ACL 2023, Toronto, Canada, 9–14 July 2023, pp. 729–743. Association for Computational Linguistics (2023). https://aclanthology.org/2023.acl-industry.70
Feng, X., Feng, X., Qin, B.: A survey on dialogue summarization: recent advances and new frontiers. arXiv preprint arXiv:2107.03175 (2021)
Ferguson, N., Guillou, L., Nuamah, K., Bundy, A.: Investigating the use of paraphrase generation for question reformulation in the FRANK QA system. arXiv preprint arXiv:2206.02737 (2022)
Gao, S., Chen, X., Ren, Z., Zhao, D., Yan, R.: Meaningful answer generation of e-commerce question-answering. ACM Trans. Inf. Syst. (TOIS) 39(2), 1–26 (2021)
Kumar, G., Henderson, M., Chan, S., Nguyen, H., Ngoo, L.: Question-answer selection in user to user marketplace conversations. In: D’Haro, L.F., Banchs, R.E., Li, H. (eds.) 9th International Workshop on Spoken Dialogue System Technology. LNEE, vol. 579, pp. 397–403. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-9443-0_35
Li, Y., et al.: Question answering for technical customer support. In: Zhang, M., Ng, V., Zhao, D., Li, S., Zan, H. (eds.) NLPCC 2018, Part I 7. LNCS (LNAI), vol. 11108, pp. 3–15. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99495-6_1
Liao, L.Y., Fares, T.: A practical 2-step approach to assist enterprise question-answering live chat. In: Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 457–468 (2021)
Lyu, Q., Zhang, H., Sulem, E., Roth, D.: Zero-shot event extraction via transfer learning: challenges and insights. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pp. 322–332 (2021)
Mao, Y., et al.: Generation-augmented retrieval for open-domain question answering. arXiv preprint arXiv:2009.08553 (2020)
Masterov, D.V., Mayer, U.F., Tadelis, S.: Canary in the e-commerce coal mine: detecting and predicting poor experiences using buyer-to-seller messages. In: Proceedings of the Sixteenth ACM Conference on Economics and Computation, pp. 81–93 (2015)
McDonald, T., et al.: Detect, retrieve, comprehend: a flexible framework for zero-shot document-level question answering. arXiv preprint arXiv:2210.01959 (2022)
Peng, B., et al.: Check your facts and try again: improving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813 (2023)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(1), 5485–5551 (2020)
Rennard, V., Shang, G., Hunter, J., Vazirgiannis, M.: Abstractive meeting summarization: a survey. arXiv preprint arXiv:2208.04163 (2022)
Samarakoon, L., Kumarawadu, S., Pulasinghe, K.: Automated question answering for customer helpdesk applications. In: 2011 6th International Conference on Industrial and Information Systems, pp. 328–333. IEEE (2011)
Shi, F., et al.: Large language models can be easily distracted by irrelevant context. arXiv preprint arXiv:2302.00093 (2023)
Vakulenko, S., Longpre, S., Tu, Z., Anantha, R.: Question rewriting for conversational question answering. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp. 355–363 (2021)
Yang, A., Liu, K., Liu, J., Lyu, Y., Li, S.: Adaptations of ROUGE and BLEU to better evaluate machine reading comprehension task. In: Choi, E., Seo, M., Chen, D., Jia, R., Berant, J. (eds.) Proceedings of the Workshop on Machine Reading for Question Answering, ACL 2018, Melbourne, Australia, 19 July 2018, pp. 98–104. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/W18-2611. https://aclanthology.org/W18-2611/
Zaib, M., Zhang, W.E., Sheng, Q.Z., Mahmood, A., Zhang, Y.: Conversational question answering: a survey. Knowl. Inf. Syst. 64(12), 3151–3195 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fetahu, B., Mehta, T., Song, Q., Vedula, N., Rokhlenko, O., Malmasi, S. (2024). Instant Answering in E-Commerce Buyer-Seller Messaging Using Message-to-Question Reformulation. In: Goharian, N., et al. Advances in Information Retrieval. ECIR 2024. Lecture Notes in Computer Science, vol 14611. Springer, Cham. https://doi.org/10.1007/978-3-031-56066-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-56066-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-56065-1
Online ISBN: 978-3-031-56066-8
eBook Packages: Computer ScienceComputer Science (R0)