Abstract
Refactoring has been reported as a helpful technique to systematically improve non-functional attributes of software. This paper evaluates the relevance of refactoring for improving usability on web applications. We conducted an experiment with two replications at different locations, with subjects of different profiles. Objects chosen for the experiment were two e-commerce applications that exhibit common business processes in today’s web usage. Through the experiment we found that half of the studied refactorings cause a significant improvement in usability. The rest of the refactorings required a post-hoc analysis in which we considered aspects like user expertise in the interaction with web applications or type of application. We conclude that, when improving quality in use, the success of the refactoring process depends on several factors, including the type of software system, context and users. We have analyzed all these aspects, which developers must consider for a better decision support at the time of prioritizing improvements and outweighing effort.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barnes S, Vidgen R (2000) WebQual: An Exploration of Web-Site Quality. In: Proceedings of the eighth European conference on information systems. pp 298–305
Barnes SJ, Vidgen R (2003) Measuring Web site quality improvements: a case study of the forum on strategic management knowledge exchange. Ind Manag Data Syst 103(5):297–309
Basili VR, Caldiera G, Rombach HD (1994) The goal question metric approach. Encycl Softw Eng 1:528–532
Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B Methodol 57:289–300
Blaikie N (2003) Analyzing quantitative data: From description to explanation. Sage Publications
Bruun A et al (2014) Active Collaborative Learning: Supporting Software Developers in Creating Redesign Proposals. In: Human-Centered Software Engineering. Springer Berlin Heidelberg, pp 1–18
Cohen L, Manion L, Morrison K (2007) Research Methods in Education
Dearden A, Finlay J (2006) Pattern languages in HCI: a critical review. Human–Comput Interact 21(1):49–102
Dig D (2011) A refactoring approach to parallelism. IEEE Softw 28(1):17–22
Distante D et al (2014) Business processes refactoring to improve usability in e-commerce applications. Electron Commer Res 14(4):1–42
Dybå T, Kampenes VB, Sjøberg DIKK (2006) A systematic review of statistical power in software engineering experiments. Inf Softw Technol 48(8):745–755
Faul F et al (2007) G* power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 39(2):175–191
Fernandez A, Insfran E, Abrahão S (2011) Usability evaluation methods for the web: a systematic mapping study. Inf Softw Technol 53(8):789–817
Fernández-Ropero M et al (2012) Quality-Driven Business Process Refactoring. In: International Conference on Business Information Systems (ICBIS 2012). pp 960–966
Fowler M, Beck K (1999) Refactoring: improving the design of existing code. Addison-Wesley
Garrido A, Rossi G, Distante D (2009) Systematic improvement of web applications design. J Web Eng 8(4):371–404
Garrido A, Rossi G, Distante D (2011) Refactoring for usability in web applications. IEEE Softw 28(3):60–67
Garrido A et al (2013) Personalized web accessibility using client-side refactoring. IEEE Internet Comput 17(4):58–66
Grissom RJ, Kim JJ (2005) Effect Sizes For Research: A Broad Practical Approach. Taylor & Francis Group
Harold E (2008) Refactoring html: improving the design of existing web applications. Addison-Wesley Professional
ISO I (2011) ISO/IEC 25010 - Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models
Jönsson P, Wohlin C (2004) An evaluation of k-nearest neighbour imputation using lIkert data. In: Proceedings - International Software Metrics Symposium. pp 108–118
Juristo, N. & Moreno, A., 2010. Basics of software engineering experimentation, Springer Publishing Company, Incorporated
Kim M, Zimmermann T, Nagappan N (2012) A Field Study of Refactoring Challenges and Benefits. In: Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering - FSE’12. ACM. p 50
Maruyama K (2007) Secure Refactoring - Improving the Security Level of Existing Code. In: ICSOFT 2007, Proceedings of the Second International Conference on Software and Data Technologies, Volume SE, Barcelona, Spain, July 22–25, 2007. pp 222–229
Moody D et al (2003) Evaluating the quality of information models: empirical testing of a conceptual model quality framework. Proceedings of the 25th International Conference on Software Engineering. pp 295–305
Murphy-Hill E, Parnin C, Black AP (2012) How we refactor, and how we know it. IEEE Trans Softw Eng 38:5–18
Nanard M, Nanard J, Kahn P (1998) Pushing Reuse in Hypermedia Design: Golden Rules, Design Patterns and Constructive Templates. In: Proceedings of the ninth ACM conference on Hypertext and hypermedia : links, objects, time and space---structure in hypermedia systems links, objects, time and space---structure in hypermedia systems - HYPERTEXT’98. ACM Press, New York, pp 11–20
Nielsen J (1999) Designing Web Usability, New Riders Publishing
Nielsen J, Tahir M (2002) Homepage usability: 50 websites deconstructed, New Riders
Olsina L et al (2008) Web applications refactoring and evaluation: a quality-oriented improvement approach. J Web Eng 7(4):258–280
Rieger M et al (2007) Refactoring for performance: an experience report. Proc Softw Evol 2:9
Shneiderman S, Plaisant C (2005) Designing the user interface, 4th edn. Pearson Addison Wesley, USA
Vakilian M et al (2012) Use, disuse, and misuse of automated refactorings. In: Proceedings - International Conference on Software Engineering. pp 233–243
Van Duyne D, Landay J, Hong J (2007) The design of sites: Patterns for creating winning web sites. Prentice Hall Professional
Van Welie M, Trætteberg H (2000) Interaction patterns in user interfaces. 7th Pattern Languages of Programs Conference, 13, p 16
Wnuk K, Gorschek T, Zahda S (2013) Obsolete software requirements. Inf Softw Technol 55:921–940
Wohlin C et al (2012) Experimentation in software engineering: an introduction. Springer
Ying M, Miller J (2013) Refactoring legacy AJAX applications to improve the efficiency of the data exchange component. J Syst Softw 86(1):72–88
Zibran MF, Roy CK (2011) A constraint programming approach to conflict-aware optimal scheduling of prioritized code clone refactoring. In: Proceedings - 11th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2011. pp 105–114
Zou Y, Zhang Q, Zhao X (2007) Improving the usability of e-commerce applications using business processes. IEEE Trans Softw Eng 33(12):837–855
Acknowledgments
This work was developed with the support of the Spanish Ministry of Science and Innovation project SMART ADAPT (TIN2013-42981-P) and was co-financed by ERDF. It also has the support of Generalitat Valenciana-funded IDEO project (PROMETEOII/2014/039).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Paolo Tonella
Alejandra Garrido and Gustavo Rossi are also at CONICET, Argentina
Appendices
Appendix 1
This appendix lists, for each of the websites used in the empirical study, the user tasks, component activities, and refactorings applied to each activity of the experiment.
Zencart-based online-store
User Task | Component Activities | Refactorings tested with the activity |
Task 1. Register to the website | 1. Access the user registration page | R1 - Improve the description of process links (having the “Login” link to navigate only to the Login page, and not to the registration) |
R2 - Split page (separating the “Login” page from the “Register” one) | ||
2. Fill in the user registration form with provided data (in particular, shipping and billing address must be the same) | R3 - Anticipate a validation activity (validating date of birth, equality of both password fields, and email format, as soon as the focus moves out of the correspondent field) | |
R4 - Change widget (using a javascript calendar to fill in the date of birth) | ||
R5 - Add a verification activity (adding a captcha at the end of the registration form) | ||
R6 - Add a “confirm and commit” activity (summarizing all entered data and presenting it in a page asking for confirmation or otherwise return to the form) | ||
3. Finalize the registration (click on submit) | ||
4. Log Out | ||
Task 2: Update user registration info | 1. Log in into the website | |
2. Update your account info: • Change billing address • Change your password • Add a second shipping address | R7 - Aggregate activities (having in one page the activities to: view or change account information, view or change entries in address book, change account password) | |
R3 - Anticipate a validation activity (anticipating the validation of: date of birth, “confirm password” checking that it is equal to the field “password”, and email format | ||
3. Finalize the process (click submit) | ||
4. Log out | ||
Task 3: Search for a number of products in the catalog of the store | 1. Log in into the website | |
2. Search for “Mouse” and check if any “Microsoft” mouse is available and visit the page of the product | R21 - Add an “assistance” activity (adding the autocomplete feature to the search textbox) | |
3. Check the technical information on the “Matrox G400” product and make sure it is compatible with Ubuntu (DO NOT USE SEARCH TOOL) | R2 - Split Page (splitting product information into different sections that can be navigated by tabs: “Description”, “Technical info”, “Shipping Costs”.) | |
4. Check the average ranking of reviews available for the “Matrox G400” product and read the review which ranks the product “2 stars”. | R8 - Introduce Information on demand (instead of having the review’s graphic at the end of the page, showing the number of reviews for each ranking value when hovering over “Reviews” at the top) | |
R9 - Turn attribute into link (turning each ranking value into a link that navigates to the page showing the reviews on the product, and specifically, to the point where reviews on that ranking value start) | ||
5. Check which are the other products available in the same category of the “Matrox G400” (DO NOT USE THE BACK button of the browser) | R10 - Provide breadcrumbs (for category) | |
6. Log out | ||
Task 4: Make a purchase with several products | 1. Enter the category “Hardware -- > Graphics Cards”, view info on the product “Matrox G200 MMS”, choose the version of the product with 16 MB of memory and add 2 items of it to your shopping cart | R11 - Move widget (changing the position of “Quantity” textbox, “Add to cart” button and the “Please Choose” group to make them visible without scrolling the page) |
2. Enter the category “DVD Movies -- > Action”, search in the list of products the titles “The Matrix Linked” and “Under Siege Linked”, “Fire Down Below Linked”, “Speed Linked”, and add them to your shopping cart, directly from the list, without viewing the details on each DVD. | R12 - Split List (diving the list of movies, organized in alphabetic order, by ranges of letters “A-D”, “E-I”, “L-O”, “P-Z”, with links to each group at the top of the page) | |
R13 - Distribute menu (Distribute the “Add selected products to cart” button to each of the product thumbnails shown in a category changing the text into “Add to shopping cart”) | ||
R14 - Keep the user up to date on the ongoing process (showing an overlay message saying that the product has actually been added to the cart) | ||
3. Enter the category “Hardware -- > Printers”, view detailed info for printer “EPSON MX14” and view the image of the printer in detail | R4 - Change widget (change the widget used to show the “Larger image” of the product with a RIA one enabling to zoom the image and see details of it in the same page) | |
4. Add two items of the EPSON MX14 printer to the shopping cart | R11 - Move widget (change the position of “Quantity” textbox, “Add to cart” button to make them visible without scrolling the page) | |
5. Check the number of items and total cost of products in the shopping cart | R15 - Improve link destination announcement (about the shopping cart) | |
6. Visit the shopping cart and remove “The Matrix Linked” DVD and add one more item of the “Under Siege Linked” DVD | R16 - Remove duplicated process links (to remove products of the shopping cart) | |
7. Start the checkout process | R17 - Make explicit the steps composing a process and the current step being executed (adding the steps of the checkout process and highlighting the current step) | |
8. Select as shipping address the second address in your address book and as shipping method the cheapest one, then continue checkout | R7 - Aggregate activities (inserting the “Change address” activity into the “Shipping information” one, by enabling the user to select one of the addresses in its address book, without the need to navigate to the address book webpage) | |
9. Add payment info by entering credit card data | R3 - Anticipate a validation activity (for credit card data form) | |
10. Go back in the process to change your shipping address to be the first of your address book | R18 - Enable the user to go back in the process steps (turning the process steps shown on top of the page to be clickable and allow navigation to each one) | |
11. Go ahead in the process to the Order Confirmation page and verify that you are purchasing the correct products and quantities | R19 - Add a summary activity (with information on the shopping cart before confirm) | |
12. Confirm the order by pushing the “Confirm the order” button | R20 - Add processing page (having a “Processing page” to inform the user that the order is being stored and inviting him to wait) | |
13. Click the OK button in the confirmation page you receive after finalizing your order |
WeBid-based online auction
User Task | Component Activities | Refactorings tested with the activity |
1. Register to the website | 1. Access the user registration page | R1 - Improve the description of process links (having the “Login” link to navigate only to the Login page, and not to the registration) |
R2 - Split page (separating the “Login” page from the “Register” one) | ||
2. Fill in the user registration form by specifying provided data | R3 - Anticipate a validation activity (validating date of birth, username already exists, equality of both password fields, and email format, as soon as the focus moves out of the correspondent field) | |
R5 - Add a verification activity (adding a captcha at the end of the registration form) | ||
3. Verify the “Terms and Conditions” and submit the registration form | R9 - Turn attribute into link (from the phrase “I agree to the terms and conditions” to the actual terms and condition statement. | |
4. Activate your user account (check email and press “Activate me”) to finalize the registration | ||
2. Update your user registration info | 1. Log in into the website | |
2. Update your account info: a. change your address b. Change your password c. Subscribe to newsletter d. Change payment method | R16 - Remove duplicated process links (so the “My control panel” menu is not duplicated) | |
R7 - Aggregate activities (having in one page the activities to: change account password, change address and basic info, subscribe to newsletter and payment info) | ||
3. Update your date of birth | R4 - Change widget (using a Javascript calendar to enable the user modify his date of birth) | |
4. Confirm changes | R6 - Add a “confirm and commit” activity | |
5. Finalize the process (click “Save changes”) | R16 - Remove duplicated process links (for the logout link) | |
6. Log out | ||
3. Search for a number of items in the auction website | 1. Log in into the website | |
2. Search for “guitar” and check if a “Fender” guitar is available | R21 - Add an “assistance” activity (adding the autocomplete feature to the search textbox) | |
3. Check the payment and shipping information on the “Fender guitar” and check (i) if the seller ships internationally (ii) if the seller accepts Cash on delivery (DO NOT USE THE SEARCH TOOL) | R2 - Split Page (diving information into different sections that can be navigated by tabs: “Item description”, “Picture gallery”, “Shipping & Payment”) | |
4. Check how many active auctions the seller of the “Fender guitar” has | R8 - Introduce information on demand (when the user hovers the mouse over the “View active auctions” link, pop-up some of the information without needing to navigate) | |
5. Navigate to the home page of the website, look for an auction on a Nikon D4 camera and access the web page showing the details on the bid | R9 - Turn attribute into link (so that the pictures of the items are clickable) | |
6. Check if the camera has a shoe mount for an external flash | R4 - Change widget (change the widget used to show the “Larger image” of the product with a RIA one enabling to zoom the image and see details of it in the same page) | |
7. Check which are the other products available in the same category of the “Nikon D4” (DO NOT USE THE BACK button of the browser) | R10 - Provide breadcrumbs | |
8. Log out | ||
4. Make a bid on a iPad | 1. Enter the category “Electronics & Photography”, search for an auction on the iPad with 16 GB of memory and visit the product details web page | R12 - Split List (diving the list of items, organized in alphabetic order, by ranges of letters “A-D”, “E-I”, “L-O”, “P-Z”, with links to each group at the top of the page) |
2. Find out the amount of the current bid in dollars. | R15 - Improve link destination announcement (where it says the unit of the price, e.g., EUR, show an image of a calculator to better announce that the link goes to a currency converter) | |
3. Place a bid for 90 EUR first, and then another for 110 EUR on the iPad 16GB | R3 - Anticipate a validation activity (anticipating the validation of the amount to be higher than the current highest bid) | |
4. Find out the status of your bid | R14 - Keep the user informed on the ongoing process (showing the “You’re the highest bidder sign”.) | |
5. Find out what an “Item watch” means by checking it in the page: Help -- > Buying | R16 - Remove duplicated process links (for the Help menu) | |
6. Add to your watch list every item in the category “Home & Garden” | R11 - Move widget R13 - Distribute Menu (for the actions “Add to watch list” and “Send to a friend”) | |
7. Log out | ||
5. Sell a DVD | 1. Suppose you want to sell a DVD into the auction website. Log into your account and proceed to sell an item | R17 - Make explicit the steps composing a process and the current step being executed (adding the steps of the process -- “Add product details”, “Specify payment and shipping info”, “Confirm auction” -- and highlighting the current step) |
2. Select the category “Music & Video” -- > “DVD” for the item you want to sell and go ahead to the next step in the process | R15 - Improve link destination announcement (replacing the text of the button from “Select category” to “Proceed to next step”.) | |
3. Specify “Speed linked” as title of the DVD you want to sell, “DVD Regional Code: 2” as item description and select “Wire Transfer” as payment method. Then complete the process | R2 - Split Page (to enter all data on the auction, Payment and shipping will be on a different page) | |
4. Go back in the process and change the category of your item to “Music & Video” -- > “HD-DVD” | R18 - Enable the user to go back in the process steps (turning the process steps shown on top of the page to be clickable and allow navigation to each one) | |
5. Change the item description by adding a picture for it (use the picture provided) | R20 - Add processing page (having a “Processing page” to inform the user that the picture is being uploaded and inviting him to wait) | |
6. Select “Paypal” as payment method, shipping fee should be 20 and specify “Seller pays shipping expenses” then submit the form | (Related to previous refactorings “Make explicit the steps composing a process” and “Split page” | |
7. Confirm your auction | R19 - Add a summary activity (showing the current version of the auction confirmation page, which shows a summary of the auction data) | |
R1 - Improve the description of process links (the page showing the summary of the auction data and asking for a confirmation will show two buttons: “Confirm auction” and “Cancel” instead of “Submit auction”.) | ||
8. Log out |
Appendix 2
Appendix 3
Rights and permissions
About this article
Cite this article
Grigera, J., Garrido, A., Panach, J.I. et al. Assessing refactorings for usability in e-commerce applications. Empir Software Eng 21, 1224–1271 (2016). https://doi.org/10.1007/s10664-015-9384-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-015-9384-6