A Reinforcement Learning Based Model for Adaptive Service Quality Management in E-Commerce Websites | Business & Information Systems Engineering Skip to main content
Springer Nature Link
Log in

A Reinforcement Learning Based Model for Adaptive Service Quality Management in E-Commerce Websites

  • Research Paper
  • Published:
Business & Information Systems Engineering Aims and scope Submit manuscript

Abstract

Providing high-quality service to all users is a difficult and inefficient strategy for e-commerce providers, especially when Web servers experience overload conditions that cause increased response time and request rejections, leading to user frustration and reduced revenue. In an e-commerce system, customer Web sessions have differing values for service providers. These tend to: give preference to customer Web sessions that are likely to bring more profit by providing better service quality. This paper proposes a reinforcement-learning based adaptive e-commerce system model that adapts the service quality level for different Web sessions within the customer’s navigation in order to maximize total profit. The e-commerce system is considered as an electronic supply chain which includes a network of basic e- providers used to supply e-commerce services for end customers. The learner agent noted as e-commerce supply chain manager (ECSCM) agent allocates a service quality level to the customer’s request based on his/her navigation pattern in the e-commerce Website and selects an optimized combination of service providers to respond to the customer’s request. To evaluate the proposed model, a multi agent framework composed of three agent types, the ECSCM agent, customer agent (buyer/browser) and service provider agent, is employed. Experimental results show that the proposed model improves total profits through cost reduction and revenue enhancement simultaneously and encourages customers to purchase from the Website through service quality adaptation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  • Aciar S, Zhang D, Simoff S, Debenham J (2007) Informed recommender: basing recommendations on consumer product reviews. IEEE Intell Syst 22(3):39–47

    Article  Google Scholar 

  • Al-Masri E, Mahmoud QH (2007) QoS-based discovery and ranking of web services. In: 16th International conference on computer communications and networks. IEEE, pp 529–534

  • Bathumalai G (2008) Self-adapting websites: mining user access logs. Dissertation, Robert Gordon University, Aberdeen

  • Belk M, Fidas C, Germanakos P, Samaras G (2015) Do human cognitive differences in information processing affect preference and performance of CAPTCHA? Int J Hum Comput Stud 84:1–18

    Article  Google Scholar 

  • Bellifemine F, Poggi A, Rimassa G (2001) JADE: a FIPA2000 compliant agent development environment. In: 5th International conference on autonomous agents. ACM, pp 216–217

  • Bhatti N, Friedrich R (1999) Web server support for tiered services. IEEE Netw 13(5):64–71

    Article  Google Scholar 

  • Brusilovsky P, Kobsa A, Nejdl W (2007) The adaptive web. Lecture notes in computer science, vol 4321. Springer, Cham, pp 325–341

    Google Scholar 

  • Chan NN, Gaaloul W, Tata S (2012) A recommender system based on historical usage data for web service discovery. Serv Oriented Comput Appl 6(1):51–63

    Article  Google Scholar 

  • Chen M, Ryu YU (2013) Facilitating effective user navigation through website structure improvement. IEEE Trans Knowl Data Eng 25(3):571–588

    Article  Google Scholar 

  • Chen CC, Huang TC, Park JJ, Yen NY (2015) Real-time smartphone sensing and recommendations towards context-awareness shopping. Multimed Syst 21(1):61–72

    Article  Google Scholar 

  • Ewing JM, Menascé DA (2009) Business-oriented autonomic load balancing for multi-tiered Web sites. In: IEEE international symposium on modeling, analysis and simulation of computer and telecommunication systems. IEEE, pp 1–10

  • Ghavamipoor H, Golpayegani SAH (2013) Comparing and applying the approach of supply chain in electronic services management. Int J Comput Inf Technol 1(2):118–136

    Google Scholar 

  • Ghavamipoor H, Golpayegani SAH (2016) QoS-aware provider selection in e-services supply chain. In: 8th International conference on information and knowledge technology. IEEE, pp 258–262

  • Ghavamipoor H, Golpayegani SAH (2017) A QoS sensitive model for e-commerce customer behavior. J Res Interact Mark 11(4):380–397

    Article  Google Scholar 

  • Harini N, Padmanabhan TR (2013) Admission control and request scheduling for secured-concurrent-available architecture. Int J Comput Appl 63(6):24–30

    Google Scholar 

  • Hong J, Suh EH, Kim J, Kim S (2009) Context-aware system for proactive personalized service based on context history. Expert Syst Appl 36(4):7448–7457

    Article  Google Scholar 

  • Lakshmi MS, Kumar SP, Janardhan M, Gayathri K (2017) Machine learning methods for refining SLA based admission control and resource allocation in cloud computing. Int J Adv Res Comput Sci 8(9):834–840

    Article  Google Scholar 

  • Larisa G, Mariia S, Andriy R (2014) Control strategy of the input stream on the online charging system in peak load moments. In: 24th International crimean conference microwave and telecommunication technology. IEEE, pp 312–313

  • Lee Y, Kozar KA (2006) Investigating the effect of website quality on e-business success: an analytic hierarchy process (AHP) approach. Decis Support Syst 42(3):1383–1401

    Article  Google Scholar 

  • Li K, Jamin S (2000) A measurement-based admission-controlled web server. In: Proceedings of the 19th annual joint conference of the IEEE computer and communications societies, vol 2, pp 651–659

  • Li YM, Wu CT, Lai CY (2013) A social recommender mechanism for e-commerce: combining similarity, trust, and relationship. Decis Support Syst 55(3):740–752

    Article  Google Scholar 

  • Lin HF (2007) The impact of Website quality dimensions on customer satisfaction in the B2C e-commerce context. Total Qual Manag Bus Excell 18(4):363–378

    Article  Google Scholar 

  • Mark K, Csaba L (2007) Analyzing customer behavior model graph (CBMG) using Markov chains. In: 11th International conference on intelligent engineering systems. IEEE, pp 71–76

  • Menascé DA (2002) QoS issues in web services. IEEE Internet Comput 6(6):72–75

    Article  Google Scholar 

  • Poggi N (2014) AUGURES: profit-aware web infrastructure management. Doctoral thesis, Polytechnic University of Catalonia

  • Poggi N, Carrera D, Gavalda R, Ayguadé E (2011) Non-intrusive estimation of QoS degradation impact on e-commerce user satisfaction. In: 10th IEEE international symposium on network computing and applications. IEEE, pp 179–186

  • Raufi B, Georgieva J, Luma A, Ismaili F, Zenuni X (2010) An adaptation algorithm for adaptive Web based systems based on link structure and document similarity. In: 9th WSEAS international conference on telecommunications and informatics. World Scientific and Engineering Academy and Society, pp 29–34

  • Rosaci D, Sarné GM (2012) A multi-agent recommender system for supporting device adaptivity in e-commerce. J Intell Inf Syst 38(2):393–418

    Article  Google Scholar 

  • Sarwar B et al (2001) Item-based collaborative filtering recommendation algorithms. In: 10th international conference on world wide web. ACM, pp 285–295

  • Schafer JB et al (2007) Collaborative filtering recommender systems. The adaptive web. Springer, Heidelberg, pp 291–324

    Google Scholar 

  • Suchacka G, Borzemski L (2013) Web server support for e-customer loyalty through QoS differentiation. In: Nguyen NT (eds) Transactions on computational collective intelligence XII. Lecture Notes in Computer Science. vol. 8240. Springer, Heidelberg

  • Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

    Google Scholar 

  • Totok A, Karamcheti V (2010) RDRP: reward-driven request prioritization for e-commerce web sites. Electron Commer Res Appl 9(6):549–561

    Article  Google Scholar 

  • Urgaonkar B (2005) Dynamic resource management in internet hosting platforms. Doctoral dissertation, University of Massachusetts Amherst

  • Urgaonkar B, Shenoy P (2005) Cataclysm: policing extreme overloads in internet applications. In: 14th International conference on world wide web. ACM, pp 740–749

  • Weiss G (1999) Multiagent systems: a modern approach to distributed artificial intelligence. MIT Press, Cambridge

    Google Scholar 

  • Yin PY, Guo YM (2013) Optimization of multi-criteria website structure based on enhanced tabu search and web usage mining. Appl Math Comput 219(24):11082–11095

    Google Scholar 

  • Yue C, Wang H (2007) Profit-aware admission control for overload protection in e-commerce web sites. In: 15th IEEE international workshop on quality of service. IEEE, pp 188–193

  • Zatwarnicki K, Zatwarnicka A (2014) The cluster-based time-aware web system. In: International conference on computer networks. Springer, pp 37–46

  • Zheng Z, Zhang Y, Lyu MR (2014) Investigating QoS of real-world web services. IEEE Trans Serv Comput 7(1):32–39

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer and Information Technology Engineering Department, Amirkabir University (Tehran Polytechnic), Tehran, Iran

    Hoda Ghavamipoor & S. Alireza Hashemi Golpayegani

Authors
  1. Hoda Ghavamipoor
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Alireza Hashemi Golpayegani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Alireza Hashemi Golpayegani.

Additional information

Accepted after two revisions by Matthias Jarke.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghavamipoor, H., Hashemi Golpayegani, S.A. A Reinforcement Learning Based Model for Adaptive Service Quality Management in E-Commerce Websites. Bus Inf Syst Eng 62, 159–177 (2020). https://doi.org/10.1007/s12599-019-00583-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12599-019-00583-6

Keywords

Search

Navigation