Explaining Recommendations: Design and Evaluation | SpringerLink
Skip to main content
Springer Nature Link
Log in

Explaining Recommendations: Design and Evaluation

  • Chapter
Recommender Systems Handbook

Abstract

This chapter gives an overview of the area of explanations in recommender systems. We approach the literature from the angle of evaluation: that is, we are interested in what makes an explanation “good”. The chapter starts by describing how explanations can be affected by how recommendations are presented, and the role the interaction with the recommender system plays w.r.t. explanations. Next, we introduce a number of explanation styles, and how they are related to the underlying algorithms. We identify seven benefits that explanations may contribute to a recommender system, and relate them to criteria used in evaluations of explanations in existing recommender systems. We conclude the chapter with outstanding research questions and future work, including current recommender systems topics such as social recommendations and serendipity. Examples of explanations in existing systems are mentioned throughout.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 37751
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 47189
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    A fifth section on mixed interaction interfaces is appended to the end of this original list.

  2. 2.

    The author does not specify which similarity metric was used, though it is likely to be a form of rating based similarity measure such as cosine similarity.

  3. 3.

    http://online.wsj.com/article_email/SB1038261936872356908.html, retrieved Feb. 12, 2009.

  4. 4.

    In [76] participants reported that they found incorrect overestimation less useful in high cost domains compared to low cost domains.

  5. 5.

    By overestimation we mean that the prediction is higher than the final or actual rating, and underestimation when the prediction is lower than it.

  6. 6.

    Here we mean the entire recommendation process, inclusive of the explanations. We note however that the evaluation of explanations in recommender systems are seldom fully independent of the underlying recommendation process.

  7. 7.

    http://www.aea.net/AvionicsNews/ANArchives/DesignDisplayOct03.pdf, retrieved Nov. 2013.

References

  1. Pandora (2006). http://www.pandora.com

  2. Nutking (2010). http://nutking.ectrldev.com/nutking/jsp/language.do?action=english

  3. Adamopoulos, P., Tuzhilin, A.: On unexpectedness in recommender systems: Or how to except the unexpected. In: Workshop on Novelty and Diversity in Recommender Systems in conjuction with Recsys (2011)

    Google Scholar 

  4. Adrissono, L., Goy, A., Petrone, G., Segnan, M., Torasso, P.: Intrigue: Personalized recommendation of tourist attractions for desktop and handheld devices. Applied Artificial Intelligence 17, 687–714 (2003)

    Article  Google Scholar 

  5. Ahn, J.W., Brusilovsky, P., Grady, J., He, D., Syn, S.Y.: Open user profiles for adaptive news systems: help or harm? In: World Wide Web (WWW), pp. 11–20. ACM Press, New York, NY, USA (2007)

    Google Scholar 

  6. Andersen, S.K., Olesen, K.G., Jensen, F.V.: HUGIN—a shell for building Bayesian belief universes for expert systems. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1990)

    Google Scholar 

  7. Backstrom, L., Sun, E., Marlow, C.: Find me if you can: Improving geographical prediction with social and spatial proximity. In: World Wide Web (WWW) (2010)

    Google Scholar 

  8. Bennett, S.W., Scott., A.C.: The Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project, chap. 19 - Specialized Explanations for Dosage Selection, pp. 363–370. Addison-Wesley Publishing Company (1985)

    Google Scholar 

  9. Bilgic, M., Mooney, R.J.: Explaining recommendations: Satisfaction vs. promotion. In: Proceedings of the Wokshop Beyond Personalization, in conjunction with the International Conference on Intelligent User Interfaces, pp. 13–18 (2005)

    Google Scholar 

  10. Billsus, D., Pazzani, M.J.: A personal news agent that talks, learns, and explains. In: Proceedings of the Third International Conference on Autonomous Agents, pp. 268–275 (1999)

    Google Scholar 

  11. Bridge, D., Kelly, J.P.: Ways of computing diverse collaborative recommendations. In: Adaptive Hypermedia and Adaptive Web-based Systems (2006)

    Book  Google Scholar 

  12. Burke, R.: Hybrid recommender systems: Survey and experiments. User Modeling and User-Adapted Interaction 12(4), 331–370 (2002)

    Article  MATH  Google Scholar 

  13. Burke, R.D., Hammond, K.J., Young, B.C.: Knowledge-based navigation of complex information spaces. In: AAAI/IAAI, Vol. 1, pp. 462–468 (1996)

    Google Scholar 

  14. Carenini, G., Mittal, V., Moore, J.: Generating patient-specific interactive natural language explanations. Proc Annu Symp Comput Appl Med Care pp. 5–9 (1994)

    Google Scholar 

  15. Chen, L., Pu, P.: Trust building in recommender agents. In: WPRSIUI in conjunction with Intelligent User Interfaces, pp. 93–100 (2002)

    Google Scholar 

  16. Chen, L., Pu, P.: Hybrid critiquing-based recommender systems. In: Intelligent User Interfaces, pp. 22–31 (2007)

    Google Scholar 

  17. Chen, L., Pu, P.: Interaction design guidelines on critiquing-based recommender systems. User Modeling and User-Adapted Interaction 3, 167–206 (2009)

    Article  MATH  Google Scholar 

  18. Cosley, D., Lam, S.K., Albert, I., Konstan, J.A., Riedl, J.: Is seeing believing?: how recommender system interfaces affect users’ opinions. In: CHI, Recommender systems and social computing, vol. 1, pp. 585–592 (2003)

    Google Scholar 

  19. Cramer, H., Evers, V., Someren, M.V., Ramlal, S., Rutledge, L., Stash, N., Aroyo, L., Wielinga, B.: The effects of transparency on perceived and actual competence of a content-based recommender. In: Semantic Web User Interaction Workshop, CHI (2008)

    Google Scholar 

  20. Cramer, H.S.M., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., Wielinga, B.J.: The effects of transparency on trust in and acceptance of a content-based art recommender. User Model. User-Adapt. Interact 18(5), 455–496 (2008)

    Article  Google Scholar 

  21. Czarkowski, M.: A scrutable adaptive hypertext. Ph.D. thesis, University of Sydney (2006)

    Google Scholar 

  22. Darlington, K.: Aspects of intelligent systems explanation. Universal Journal of Control and Automation 1, 40–51 (2013)

    Article  Google Scholar 

  23. Doyle, D., Tsymbal, A., Cunningham, P.: A review of explanation and explanation in case-based reasoning. Tech. rep., Department of Computer Science, Trinity College, Dublin (2003)

    Google Scholar 

  24. Erlich, K., Kirk, S., Patterson, J., Rasmussen, J., Ross, S., Gruen, D.: Taking advice from intelligent systems: The double-edged sword of explanations. In: Intelligent User Interfaces (2011)

    Book  Google Scholar 

  25. Felfernig, A., Gula, B.: Consumer behavior in the interaction with knowledge-based recommender applications. In: ECAI 2006 Workshop on Recommender Systems, pp. 37–41 (2006)

    Google Scholar 

  26. Felfernig, A., Teppan, E., Gula, B.: Knowledge-based recommender technologies for marketing and sales. Int. J. Patt. Recogn. Artif. Intell. 21, 333–355 (2007)

    Article  Google Scholar 

  27. Fogg, B., Marshall, J., Kameda, T., Solomon, J., Rangnekar, A., Boyd, J., Brown, B.: Web credibility research: A method for online experiments and early study results. In: CHI 2001, pp. 295–296 (2001)

    Google Scholar 

  28. Fogg, B.J., Soohoo, C., Danielson, D.R., Marable, L., Stanford, J., Tauber, E.R.: How do users evaluate the credibility of web sites?: a study with over 2,500 participants. In: Designing for User Experiences (DUX), no. 15 in Focusing on user-to-product relationships, pp. 1–15 (2003)

    Google Scholar 

  29. Gedikli, F., Jannach, D., Ge, M.: How should I explain? A comparison of different explanation types of recommender systems. International Journal of Human-Computer Studies 72(4), 367–382 (2014)

    Article  Google Scholar 

  30. Green, S., Lamere, P., Alexander, J., Maillet, F.: Generating transparent, steerable recommendations from textual descriptions of items. In: Recommender Systems Conference (2009)

    Book  Google Scholar 

  31. Gretarsson, B., O’Donovan, J., Bostandjiev, S., Hall, C., Höllerer, T.: Smallworlds: Visualizing social recommendations. Computer Graphics Forum 29(3), 833–842 (2010)

    Article  Google Scholar 

  32. Guy, I., Ronen, I., Wilcox, E.: Do you know? recommending people to invite into your social network. In: International Conference on Intelligent User Interfaces, pp. 77–86 (2009)

    Google Scholar 

  33. Hance, E., Buchanan, B.: Rule-based expert systems: the MYCIN experiments of the Stanford Heuristic Programming Project. Addison-Wesley (1984)

    Google Scholar 

  34. Häubl, G., Trifts, V.: Consumer decision making in online shopping environments: The effects of interactive decision aids. Marketing Science 19, 4–21 (2000)

    Article  Google Scholar 

  35. Herlocker, J.L., Konstan, J.A., Riedl, J.: Explaining collaborative filtering recommendations. In: ACM conference on Computer supported cooperative work, pp. 241–250 (2000)

    Google Scholar 

  36. Hingston, M.: User friendly recommender systems. Master’s thesis, Sydney University (2006)

    Google Scholar 

  37. Hu, R., Pu, P.: Helping users perceive recommendation diversity. In: Workshop on Novelty and Diversity in Recommender Systems in conjunction with Recsys (2011)

    Google Scholar 

  38. Hunt, J.E., Price, C.J.: Explaining qualitative diagnosis. Engineering Applications of Artificial Intelligence 1(3), Pages 161–169 (1988)

    Article  Google Scholar 

  39. Krulwich, B.: The infofinder agent: Learning user interests through heuristic phrase extraction. IEEE Intelligent Systems 12, 22–27 (1997)

    Google Scholar 

  40. Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., Wong, W.K.: Conference on visual languages and human-centric computing. In: Too Much, Too Little, or Just Right? Ways Explanations Impact End Users Mental Models (2013)

    Google Scholar 

  41. Lacave, C., Diéz, F.J.: A review of explanation methods for bayesian networks. The Knowledge Engineering Review 17:2, 107–127 (2002)

    Google Scholar 

  42. Lacave, C., Diéz, F.J.: A review of explanation methods for heuristic expert systems. The Knowledge Engineering Review 17:2, 107–127 (2004)

    Google Scholar 

  43. Lewis, C., Rieman, J.: Task-centered user interface design: a practical introduction. University of Colorado (1994)

    Google Scholar 

  44. Lopez-Suarez, A., Kamel, M.: Dykor: a method for generating the content of explanations in knowledge systems. Knowledge-based Systems 7(3), 177–188 (1994)

    Article  Google Scholar 

  45. McCarthy, K., Reilly, J., McGinty, L., Smyth, B.: Thinking positively - explanatory feedback for conversational recommender systems. In: Proceedings of the European Conference on Case-Based Reasoning (ECCBR-04) Explanation Workshop, pp. 115–124 (2004)

    Google Scholar 

  46. McGinty, L., Reilly, J.: On the evolution of critiquing recommenders. In: F. Ricci, L. Rokach, B. Shapira, P.B. Kantor (eds.) Recommender Systems Handbook, pp. 547–576. Springer US (2011)

    Google Scholar 

  47. McGinty, L., Smyth, B.: Comparison-based recommendation. Lecture Notes in Computer Science 2416, 575–589 (2002)

    Article  Google Scholar 

  48. McNee, S.M., Lam, S.K., Konstan, J.A., Riedl, J.: Interfaces for eliciting new user preferences in recommender systems. User Modeling pp. pp. 178–187 (2003)

    Google Scholar 

  49. McNee, S.M., Riedl, J., Konstan, J.A.: Being accurate is not enough: How accuracy metrics have hurt recommender systems. In: Extended Abstracts of the 2006 ACM Conference on Human Factors in Computing Systems (CHI 2006) (2006)

    Google Scholar 

  50. McSherry, D.: Explanation in recommender systems. Artificial Intelligence Review 24(2), 179–197 (2005)

    Article  MATH  Google Scholar 

  51. Nagulendra, S., Vassileva, J.: Providing awareness, understanding and control of personalized stream filtering in a p2p social network. In: Conference on Collaboration and Technology (CRIWG) (2013)

    Google Scholar 

  52. Nguyen, T.T., Kluver, D., Wang, T.Y., Hui, P.M., Ekstrand, M.D., Willemsen, M.C., Rield, J.: Rating support interfaces to improve user experience and recommender accuracy. In: Recommender Systems Conference (2013)

    Book  Google Scholar 

  53. Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: ACM CHI’90, pp. 249–256 (1990)

    Google Scholar 

  54. Ohanian, R.: Construction and validation of a scale to measure celebrity endorsers’ perceived expertise, trustworthiness, and attractiveness. Journal of Advertising 19:3, 39–52 (1990)

    Google Scholar 

  55. O’Sullivan, D., Smyth, B., Wilson, D.C., McDonald, K., Smeaton, A.: Improving the quality of the personalized electronic program guide. User Modeling and User-Adapted Interaction 14, pp. 5–36 (2004)

    Article  Google Scholar 

  56. Papadimitriou, A., Symeonidis, P., Manolopoulos, Y.: A generalized taxonomy of explanation styles for traditional and social recommender systems. Data Mining and Knowledge Discovery 24, 555–583 (2012)

    Article  Google Scholar 

  57. Pazzani, M.J.: A framework for collaborative, content-based and demographic filtering. Artificial Intelligence Review 13, 393–408 (1999)

    Article  Google Scholar 

  58. Pizzato, L., Rej, T., Akehurst, J., Koprinska, I., Yacef, K., Kay, J.: Recommending people to people: the nature of reciprocal recommenders with a case study in online dating. User Modeling and User-Adapted Interaction 23, 447–488 (2013)

    Article  Google Scholar 

  59. Pu, P., Chen, L.: Trust building with explanation interfaces. In: IUI’06, Recommendations I, pp. 93–100 (2006)

    Google Scholar 

  60. Pu, P., Chen, L.: Trust-inspiring explanation interfaces for recommender systems. Knowledge-based Systems 20, 542–556 (2007)

    Article  MathSciNet  Google Scholar 

  61. Pu, P., Faltings, B., Chen, L., Zhang, J., Viappiani, P.: Usability guidelines for product recommenders based on example critiquing research. In: F. Ricci, L. Rokach, B. Shapira, P.B. Kantor (eds.) Recommender Systems Handbook, pp. 547–576. Springer US (2011)

    Google Scholar 

  62. Rafter, R., Smyth, B.: Conversational collaborative recommendation - an experimental analysis. Artif. Intell. Rev 24(3–4), 301–318 (2005)

    Article  Google Scholar 

  63. Reilly, J., McCarthy, K., McGinty, L., Smyth, B.: Dynamic critiquing. In: P. Funk, P.A. González-Calero (eds.) ECCBR, Lecture Notes in Computer Science, vol. 3155, pp. 763–777. Springer (2004)

    Google Scholar 

  64. Ricci, F.: Mobile recommender systems. Information Technology & Tourism 12.3, 205–231 (2010)

    Article  Google Scholar 

  65. Said, A., Bellogin Kouki, A., de Vries, A.P., Kille, B.: Information Retrieval And User-Centric Recommender System Evaluation. In: Extended Proceedings of The 21st Conference on User Modeling, Adaptation and Personalization (UMAP’13), http://ceur-ws.org/Vol-997/umap2013_project_3.pdf CEUR (2013). URL http://oai.cwi.nl/oai/asset/21389/21389B.pdf

  66. Shani, G., Rokach, L., Shapira, B., Hadash, S., Tangi, M.: Investigating confidence displays for top-n recommendations. Journal of the American Society for Information Science and Technology 64, 2548–2563 (2013)

    Article  Google Scholar 

  67. Sharma, A., Cosley, D.: Do social explanations work? studying and modeling the effects of social explanations in recommender systems. In: World Wide Web (WWW) (2013)

    Google Scholar 

  68. Sinha, R., Swearingen, K.: The role of transparency in recommender systems. In: Conference on Human Factors in Computing Systems, pp. 830–831 (2002)

    Google Scholar 

  69. Swearingen, K., Sinha, R.: Interaction design for recommender systems. In: Designing Interactive Systems, pp. 25–28 (2002)

    Google Scholar 

  70. Symeonidis, P., Nanopoulos, A., Manolopoulos, Y.: Justified recommendations based on content and rating data. In: WebKDD Workshop on Web Mining and Web Usage Analysis (2008)

    Google Scholar 

  71. Tanaka-Ishii, K., Frank, I.: Multi-agent explanation strategies in real-time domains. In: 38th Annual Meeting on Association for Computational Linguistics, pp. 158–165 (2000)

    Google Scholar 

  72. Thompson, C.A., Göker, M.H., Langley, P.: A personalized system for conversational recommendations. J. Artif. Intell. Res. (JAIR) 21, 393–428 (2004)

    Google Scholar 

  73. Tintarev, N.: Explaining recommendations. In: User Modeling, pp. 470–474 (2007)

    Google Scholar 

  74. Tintarev, N., Dennis, M., Masthoff, J.: Adapting recommendation diversity to openness to experience: A study of human behaviour. In: UMAP (2013)

    Google Scholar 

  75. Tintarev, N., Masthoff, J.: Effective explanations of recommendations: User-centered design. In: Recommender Systems, pp. 153–156 (2007)

    Google Scholar 

  76. Tintarev, N., Masthoff, J.: Over- and underestimation in different product domains. In: Workshop on Recommender Systems associated with ECAI (2008)

    Google Scholar 

  77. Tintarev, N., Masthoff, J.: Personalizing movie explanations using commercial meta-data. In: Adaptive Hypermedia (2008)

    Google Scholar 

  78. Tintarev, N., Masthoff, J.: Evaluating the effectiveness of explanations for recommender systems: Methodological issues and empirical studies on the impact of personalization. User Modeling and User-Adapted Interaction 22, 399–439 (2012)

    Article  Google Scholar 

  79. Verbert, K., Parra, D., Brusilovsky, P., Duval, E.: Visualizing recommendations to support exploration, transparency and controllability. In: Proceedings of the 2013 International Conference on Intelligent User Interfaces, IUI ‘13, pp. 351–362. ACM, New York, NY, USA (2013). DOI 10.1145/2449396.2449442

  80. Victor, P., Cock, M.D., Cornelis, C.: Trust and recommendations. In: F. Ricci, L. Rokach, B. Shapira, P.B. Kantor (eds.) Recommender Systems Handbook, pp. 547–576. Springer US (2011)

    Google Scholar 

  81. Vig, J., Sen, S., Riedl, J.: Tagsplanations: Explaining recommendations using tags. In: Intelligent User Interfaces (2009)

    Google Scholar 

  82. Wang, W., Benbasat, I.: Recommendation agents for electronic commerce: Effects of explanation facilities on trusting beliefs. Journal of Management Information Systems 23, 217–246 (2007)

    Article  Google Scholar 

  83. Wärnestål, P.: Modeling a dialogue strategy for personalized movie recommendations. In: Beyond Personalization Workshop, pp. 77–82 (2005)

    Google Scholar 

  84. Wärnestål, P.: User evaluation of a conversational recommender system. In: Proceedings of the 4th Workshop on Knowledge and Reasoning in Practical Dialogue Systems, pp. 32–39 (2005)

    Google Scholar 

  85. Webster, A., Vassileva, J.: The keeup recommender system. In: Recsys (2007)

    Google Scholar 

  86. Wick, M.R., Thompson, W.B.: Reconstructive expert system explanation. Artif. Intell. 54(1–2), 33–70 (1992)

    Article  Google Scholar 

  87. Ye, L., Johnson, P., Ye, L.R., Johnson, P.E.: The impact of explanation facilities on user acceptance of expert systems advice. MIS Quarterly 19(2), 157–172 (1995)

    Article  Google Scholar 

  88. Yee, K.P., Swearingen, K., Li, K., Hearst, M.: Faceted metadata for image search and browsing. In: ACM Conference on Computer-Human Interaction (2003)

    Book  Google Scholar 

  89. Zheng, V.W., Zheng, Y., Xie, X., Yang, Q.: Collaborative location and activity recommendations with GPS history data. In: World Wide Web (WWW) (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Aberdeen, AB24 3UE, Aberdeen, UK

    Nava Tintarev & Judith Masthoff

Authors
  1. Nava Tintarev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Judith Masthoff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nava Tintarev .

Editor information

Editors and Affiliations

  1. Faculty of Computer Science, Free University of Bozen-Bolzano, Bolzano - Bozen, Italy

    Francesco Ricci

  2. Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, Israel

    Lior Rokach

  3. Ben-Gurion University of the Negev, Beer-Sheva, Israel

    Bracha Shapira

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media New York

About this chapter

Cite this chapter

Tintarev, N., Masthoff, J. (2015). Explaining Recommendations: Design and Evaluation. In: Ricci, F., Rokach, L., Shapira, B. (eds) Recommender Systems Handbook. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-7637-6_10

Download citation

  • DOI: https://doi.org/10.1007/978-1-4899-7637-6_10

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4899-7636-9

  • Online ISBN: 978-1-4899-7637-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation