Assessing the Understandability of Attack-Defense Trees: an Empirical Study - Supplementary Material

The present folder contains the replication package for the paper 
"Assessing the Understandability of Attack-Defense Trees: an Empirical Study" by Giovanna Broccia, Alessio Ferrari, Maurice ter Beek, Alberto Lluch Lafuente and Paola Spoletini.

In particular, the folder contains the following sub-folders:
1.    Recruiting e-mail
The folder contains the e-mail sent to users to participate in the study that provides all the information to complete it in all its phases, the link to the consent form to be signed, the link to the pre-test questionnaire, the link to the test, and the link to the post-test questionnaire. By following the instruction and the links it is possible to take part in the study. 
The folder contains as well the Consent Form document.
2.    Training video
The folder contains the training video on attack-defense trees.
3.    Pre-test questionnaire
The folder contains the PDF (not editable) version of the pre-test questionnaire. The original (editable) version of the questionnaire is available at the following link https://docs.google.com/forms/d/e/1FAIpQLSem5e3YBTX0n15X4nYTEd7hRsnOch8ArCy-KOPjVXEm8Evrig/viewform 
4.    ADT test 
The folder contains the PDF (not editable) version of the test, composed of 4 different phases. To access the editable version of the test, please check the recruiting e-mail which gives participants all the information to access the test.
5.    Post-test questionnaire
The folder contains the PDF (not editable) version of the post-test questionnaire. The original (editable) version of the questionnaire is available at the following link https://docs.google.com/forms/d/e/1FAIpQLSe-t7faBICZWWP746cxH1knCGO8jnDsCsZPz0qXP-2LUdzhOQ/viewform 
6.    Data analysis
The folder contains the following files:
- Pre-test-Questionnaire-answers.csv: raw data from the pre-test questionnaire (not processed)
- Post-test-Questionnaire-answers.csv: raw data from the post-study questionnaire (not processed)
- ADT-test-results.csv: raw data from the test phase and the post-study questionnaire
- adt-data-analysis-SEFM.html: results visualised through an html document
- adt-data-analysis-SEFM.Rmd: Rmd file with R code to generate adt-data-analysis-SEFM.html

Attack-Defense Trees are a graphical notation used to evaluate the security of a system. While the quality of this notation has been primarily assessed quantitatively, its understandability is often mentioned as a key factor for its success still it has never been tested with an empirical evaluation. 
To assess the quality of such a notation in terms of understandability and user acceptance, we conducted an empirical user study. The study focuses on performance-based variables and perception-based variables, aiming to evaluate the relationship between these measures and how they might impact the practical use of the notation.
The performance-based variables are assessed through a test, while the perception-based variables are assessed through a post-test questionnaire.
The present folder contains the instruction and the material to perform the study (sub-folder 1 Recruiting e-mail), the not-editable document presenting the pre and post-test questionnaires and the test (sub-folders 3. Pre-test questionnaire, 4. ADT test, 5. Post-test questionnaire), and all the data to check the results presented in the paper (sub-folder 6. Data analysis).