Keywords

1 Introduction

The experience presented is part of the PhD project “Framework for the Design of Inclusive Computational Applications Related to The Achievement of Emotional Intelligence Skills in Children with Autism Spectrum Disorder -ASD” which contemplates the design of methodological and technological models for psycho-educational intervention that allows the increase of levels of emotional intelligence expected in children with such disability.

The U.S. Center for Disease Control and Prevention (CDC) recognizes many ways to maximize an autistic child’s ability so that he or she can grow and learn new skills. Generally, these treatments are divided into several categories, such as the Behavioral and Communication Approach, as a principle for the development of behavioral and communication skills [1]. However, these treatments include behavioral and communication therapies, skills development, or medications to control symptoms, but there are few cases where you have deepened the design of models for the use of inclusive technologies that enhance your skills and competencies.

One of the current drawbacks in the treatment of children with ASD is the traditional management of emotions by their therapists with training in aspects such as collaborative learning, social adaptation, decision-making, the ability to face conflicts, i.e. the management of emotional intelligence; but linked to this practice it is necessary to explore other alternatives related to strengthening skills without applying instruments in an invasive way or the use of computational tools that stimulate those competencies in a more natural way. One of these aspects to explore is the characteristic of how most children with ASD show interest in images (pictograms), from the notion of physical representation that is then configured as an idea in the work of these people.

That is why thinking about the design of inclusive computational applications that allow the integration of some pictographic elements into the work processes of children with ASD, can result in a significant improvement in the way they acquire skills for emotional intelligence [2], and therefore the question to be solved is to what extent can the use of inclusive computational applications facilitate the implementation of therapeutic activities that facilitate the strengthening of the levels of emotional and social intelligence in children with ASD and how this would allow to propose a framework for the development of inclusive applications aimed at the treatment of this disorder?

This is why article relates to the heuristic assessment performed on a group of computational tools made for mobile devices (APPs) included in the Apple Store and Play Store databases mainly; all this framed in a case study with children suffering from low levels of autism and with which the positive variation of their emotional states was required through the use of these tools.

People with Autism have Special Educational Needs (SEN) who until recently were only worked by a traditional methodology, but at present Information and Communication Technologies (ICT) make more and more resources adapted to people with ASD with the aim of improving virtually any area of development and their basic competences [3]. Thus, it is necessary to validate its relevance and levels of usability based on its purpose within a formal treatment for the afore mentioned disorder.

2 About Autism Spectrum Disorder

The Autism is part of permanent neurodevelopmental disorders, in which areas related to social interaction, communication, behavior and interest among others deteriorate. According to [4] the educational environment the affective dimension of learning processes is given importance, however, the emotional aspects in education remain a complex challenge today. The emotion consists of three components: Neurophysiological, Behavioral and Cognitive. The neurophysiological component manifests itself in aspects such as breathing, sweating and hypertension, which, while involuntary responses that the individual cannot control, clarifies that if they can be prevented by appropriate techniques. The behavioral component is related to facial expressions, nonverbal language, tone of voice and body movements, among others. Unlike the neurophysiological component, these expressions are controllable and provide quite precise signals about the emotional state of the person. The cognitive component is the one that relates to feelings, because fear, anguish and anger, among other emotions are expressed in this component.

The diversity of hypotheses about the nature of autism disorder over the past few decades, all of which are more rooted in the cause than on the underlying mental processes, has greatly limited the effectiveness of the different treatments applied to “rehabilitation” [5].

Advances in this type of research, relate three (3) types of autism according to the affectation of your neurological system:

  1. 1.

    TEA Level 1: No intellectual developmental disorder and requiring assistance.

  2. 2.

    TEA Level 2: With affectation of verbal and nonverbal social communication or abnormal response to the social approaches of others. Requires essential support.

  3. 3.

    TEA Level 3: Severe deficit in verbal and nonverbal social communication skills that cause severe functioning disabilities; very limited initiation of social interactions and minimal response to the social approaches of others. It requires support in an especially important and constant way.

The use of technologies to particularly improve and stimulate children’s communication with ASD has increased exponentially in recent times. These tools in therapeutic contexts enable a generalization of the behavior towards natural contexts of the child [6]. Therefore, it is intended to verify in this case study whether the use of specialized software and mobile devices allows children diagnosed with level 1 ASD to advance with their treatment, outside the clinical field, being able to use it at home/school to communicate with their close social links; therefore, this study will be designed using human-computer interaction models.

Any type of treatment that is used, analyzes the behavior of the child with ASD and according to the results the program of integral educational intervention is elaborated. It should be borne in mind that a person with ASD generally manifests deep and complex alterations in the area of communication, both verbal and non-verbal, presenting absence of communicative intent or alterations in the use of language. Therefore, within non-verbal communication, it is necessary to distinguish between instrumental acts, natural gestures and Alternative Communication Systems (SACs). For this project, we focus exclusively on alternative systems of communication, which are the skills of emotional and social intelligence that is intended to be analyzed and for which a later solution with a focus on interaction would be developed human-computer [1].

3 The Research Problem

Having understood that emotional and social management is very complex in cases of children with Autism Spectrum Disorder, where each case is different from the others, the developments and implementations of computational solutions to support treatments of this type of condition have been experienced in isolation to specialized clinical treatments [8].

Generally, these treatments are divided into several categories including the Focus on Behavior and Communication as the primary framework for behavioral and communication skills development. However, these treatments include behavioral and communication therapies, skills development, or medications to control symptoms, but there are few cases where you have deepened the design of technology use models inclusive to enhance their skills and competencies.

That is why thinking about the design of inclusive computational applications that allow the integration of some pictographic elements into the work processes of children with ASD, can result in a significant improvement in the way they acquire skills for emotional intelligence, and therefore the question is to what extent the use of inclusive computational applications can facilitate the implementation of therapeutic activities that facilitate the strengthening of intelligence levels emotional and social in children with ASD and how does this allow us to propose a framework for the development of inclusive applications aimed at the treatment of this disorder?

4 Precedent

The heuristic evaluation is based on a case study conducted with girls approximately 7 years old, which have been diagnosed with levels one of Autism, which are part of treatments based on behavioral and recreational therapies that did not include the use of computational tools. This explores and determines the characteristics of computational applications that allow the management of pictograms as an emotional and behavioral management alternative, taking as a reference the user-centered design models (DCU).

At this stage, you get a group of approximately fifty mobile apps that meet the expected functionality:

  1. 1.

    To be a communicative-linguistic tool. These applications favor body language development and self-recognition.

  2. 2.

    Be social and emotional tools. Designed for users to enjoy and learn when interacting and playing autonomously with the specific browser or application.

  3. 3.

    Be cognitive tools. They are tools with which you can design personalized and adapted educational activities that can be used anywhere, in addition to favoring the social integration of the user.

For the development of this aspect, it uses the exploration of the state of the art related to the existence of computational applications focused on supporting ASD treatments, discriminating between various factors such as: Manufacturer, Application Description, Type of Licensing, Technological Characteristics, Application Context. Finally, 12 of these tools are defined according to criteria of ease of use, ease of access and download, documentation and aesthetics.

Table 1. Computational applications for study
Full size table

It is emphasized that the purpose of the case study was to verify by applying models of human-computer interaction whether the use of specialized software, and through mobile devices, allows progress in the results of treatment of children with Autism Spectrum Disorder –ASD developing some emotional and social skills such as self-recognition and social performance of children. Con this purpose, the case study is carried out in the context of the City of Popayán (Colombia), where autism is the seventh type of disability most found in early childhood educational institutions, as reported in the last decade by the National Administrative Department of Statistics of Colombia –DANE [10].

In this case, two test units were used, corresponding to two nine-year-old girls who have previously been diagnosed with Autism Level 1 and who currently develop their treatment with expert people, a physiotherapist, or a psychologist. The first girl is part of the Leonardo Davinci Pedagogical Center, which is a private educational institution, which provides mixed school education at the levels of preschool, elementary, secondary school, cyclic baccalaureate for adults and technical work by competencies. This center is in the municipality of Popayán (Cauca-Colombia), and aims to generate a new pedagogical and social approach through educational innovation. The second case identified is a person who also has the professional accompaniment requirements that are required for research.

4.1 Data and Evidence Collection

For the implementation of the sessions of application of computational tools within the treatment of children with ASD, the documentary preparation of permits and consents should have been carried out aimed at parents of autistic children, the design of didactic activities that made use of the collected applications and the organization and enlistment of both physical spaces and tools for the taking of evidence and results to be collected.

For the collection of analytical information, the adaptation and application of conceptual tools already designed by professionals’ experts in the identification and study of autism spectrum disorder –ASD is used. In this sense, the following formats are defined to be applied to each of the selected analysis units:

  1. 1.

    Instrument 1 – Informed consent model for parents.

  2. 2.

    Instrument 2 – Clinical-therapeutic approach.

  3. 3.

    Instrument 3 – Technological approach -study usability of inclusive applications for ASD.

Similarly, the most appropriate space for the conduct of the case study is defined by applying the TEACCH Model (“Treatment and Education of Autistic and related Communication Handicapped Children”), through the application of the prepared activities, the identification in the child of the levels of autonomy for carrying out activities involving the use of emotional skills, while recognizing the difficulties of communication and understanding of the language [11].

Tasks for children with ASD who begin working with this methodology are considered complementary to established therapies where they must follow instructions, use materials and everything defined for emotional or skills management Social (Fig. 1).

Fig. 1.
figure 1

Model TEACCH

Full size image
Fig. 2.
figure 2

Distribution of artifacts in physical space

Full size image

An alternative to TEACH has been the use of NFC (“Near Field Communication”) software applications for children with functional diversity that, through their use on mobile devices, use animated pictograms for the representation of a person’s day-to-day activities, which seek to promote communication, allow to plan, organize and anticipate certain activities.

It is hoped to validate whether these tools contribute to conventional therapies and promote better social interaction. The activities of the applications are intended to be developed in a tripartite way, i.e. the interaction between the child, new technologies and the professional or tutor will be encouraged and is sought to be transferable to different areas of reference of the child, mainly family and the educational center in which he/she is located [12]. For this purpose, it is considered the need to analyze what characteristics should be, functional and non-functional, which should have an inclusive computational application that is effectively used in the treatment of autism.

The purpose of the case study is to verify by applying human-computer interaction models whether the use of specialized software, and through mobile devices, allows progress in the results of treatment of children with Autism Spectrum Disorder –ASD developing some emotional and social skills such as self-recognition and social performance of children.

In this step, the case study was designed for two (2) analysis units, which correspond to seven-year-old girls who have been previously diagnosed with level 1 of autism and who are accompanied by professionals in phono-audiology and physiotherapy. During this activity, the best physical distribution of the technological elements and equipment to be used during the study should be designed, so that during experimental activities with autistic girls they could use the inclusive applications while we could record their emotional changes through facial recognition.

The selected distribution of the elements was:

  1. 1.

    Workbench: On which is located a digital or mobile tablet where the applications selected for the study have been loaded in advance.

  2. 2.

    Chairs: Where the child with ASD is located and the person who runs the activity. An additional chair may also be available for the person collecting the information (observation sheets).

  3. 3.

    Filming Camera: For the capture of facial expressions of the child during the development of activity and the identification of expressed emotional changes.

5 Methodology

The word heuristic comes etymologically from the Greek word “euriskein” that comes from “eureka”, a word that means find or find. Based on the above definition, the Computer Person Interaction (IPO, in Spanish) presents the Heuristic Evaluation (HE) as a method of evaluating usability by inspection that must be carried out by expert evaluators based on principles (called “heuristics”) previously established. As a usability evaluation technique, the EH aims to measure the quality of the interface of any interactive system in relation to its ease to be learned and used by a certain group of users in a given context of use [ISO98, UNET06].

The Heuristic, also called the Heuristic Principle or heuristic criterion, tries to apply conversational norms to the interaction between a person and a system: its objective is to try to create a “communicational bridge” in which both the person and the system understand and work together in pursuit of a goal to achieve. These general empirical rules are used within the planning of an HE as a starting point for the creation of an item checklist that will later be used by the expert evaluator within the implementation of the evaluation. In this way, these general rules are appropriate to each specific case of evaluation to reflect in the items to be evaluated the nature and type of interface to be evaluated and the context of its use.

The usefulness of the heuristic evaluation for the applications used in this part of the project seeks to identify the most relevant functional and design characteristics of these applications and that should be taken into account for a possible improvement within the future construction of a new inclusive application that is adapted to the treatment of autism spectrum disorder (Fig. 3).

Fig. 3.
figure 3

Nielsen’s usability heuristics

Full size image

5.1 Pre-assessment Activities Carried Out

In general, activities were oriented as follows:

Theoretical Support and Systematic Review of Literature.

The concepts of emotional intelligence skills, especially Self-Knowledge and Social Skills, were explored; the most relevant features of Autism Spectrum Disorder; what treatment and education programs exist and their level of effectiveness; Notions of User-Centered Design (DCU) and Accessibility; Typology of existing mobile applications for the treatment of autism; and finally metrics and heuristics that may exist to assess the usability of inclusive applications.

This phase included searching, exploring functional features and selecting inclusive app with authoritative child pictogram handling, defining usability standards in software engineering, and finding a measurement tool (no invasive) of emotional changes and software usability.

Exploring Inclusive Applications for ASD.

For this, an app search was performed on the Apple Store and Play Store (Table 1) databases about software that was based on autism knowledge and also presented treatment alternatives based on the management of pictograms for the construction of communication structures. Post-this collection had information relevant to each app such as the type of license, a description of the app’s functionality, and an explanation of why it was selected.

Definition of Heuristic Assessment Metrics.

At this stage, the usability criteria that inclusive applications that are applied within autism treatment should be defined. These criteria were:

  1. 1.

    Ease of use

  2. 2.

    Documentation included for the app

  3. 3.

    Aesthetic

  4. 4.

    Operation

  5. 5.

    Ease of access to the tool

With the criteria already defined, an instrument was built that would allow heuristic evaluation, based on expert criteria, to be applied to the list of apps selected in the previous activity (Table 2).

Table 2. Heuristic evaluation format
Full size table

The instrument built in MS Excel for heuristic evaluation applies a formula to calculate the usability percentage (UP) of each of the applications analyzed.

5.2 Development of Heuristic of Inclusive App for ASD

This application evaluation was done from the look of two profiles in the project: Expert and User. The first consisted of expert evaluation in User-Centered Design and Mobile App Developers. The second type of evaluator consisted of the evaluation of the app according to the operation of the application by the user, i.e. the autistic child (Table 3).

Table 3. Expert groups in heuristic assessment
Full size table

Each of the experts had a mobile device (Tablet or Smart Phone) that had the previous installation of the applications to be evaluated and a personal computer with the heuristic review instrument for inclusive App designed for the project.

Each expert analyzed the mobile applications and delivered the following ratings (Table 4, Figs. 4 and 5):

Table 4. Heuristic evaluation results
Full size table
Fig. 4.
figure 4

Inclusive app valuation analysis

Full size image
Fig. 5.
figure 5

Major design elements

Full size image

This heuristic evaluation of inclusive applications was based on the definition of usability metrics of selected applications based on the most relevant standards that refer to software usability, such as ISO/IEC 9241-11, 9126-1, ISO 13407, 14598-1, 14915-1, ISO/TR 18529, 16982.

Also, usability heuristics for child learning applications are defined by classifying the set of heuristics into three categories:

  1. 1.

    Nielsen usability.

  2. 2.

    Usability for children.

  3. 3.

    Usability for learning.

Subsequently, for the evaluation of mobile applications, a usability assessment instrument is designed for the selected applications, which collects the results of their use during working sessions with the analysis units and selects those that best perform from the approach of usability and user-centered design, which in our case are children with the degree 1 of autism.

The formula for the calculation of the usability percentage achieved by each APP is as follows:

$$ PU = \frac{{\mathop \sum \nolimits_{i = 1}^{i = ncc} \left( {vc*re} \right)}}{{\mathop \sum \nolimits_{i = 1}^{i = ncc} \left( {cc*rc} \right)}}*100 $$

Where:

  • vc: Criterion value

  • re: Relevance of evaluation

  • rc: Relevance of criterion

  • cc: Number of criteria evaluated

The selection criteria for the most relevant apps were: Ease of use, Documentation, Aesthetics, Operability and Ease of Access.

The state of the art made it possible to identify design models of inclusive applications that could be applied or adapted for the development of applications focused on the treatment of autism. In this sense, there are some generic models that can be used in this purpose and another that was created exclusively for inclusive developments. The methods explored were:

  1. 1.

    MDA Method: Containing the necessary steps to define the Mechanics, Dynamics and Aesthetics of a user-centered development.

  2. 2.

    6D Method: Organized in generic design stages of computational applications that can be adapted to inclusive development through the stages of: Problem Description, Solution Definition, Solution Design, Solution Development, Debugging and testing and finally the Documentation.

  3. 3.

    MPIu+a method: This is the only method contemplated that was created for the design of inclusive applications. Its stages are: Requirements Analysis, Design, Prototype Implementation, Launch and Evaluation.

  4. 4.

    Gamification Method: Great value for adding playful features to inclusive development. In this case you can use Gamification Canvas or Octalisys models and instruments for the design of activities.

5.3 Recognition of Emotional Changes

The arrangement of a controlled environment was considered for the performance of this activity. In this case, for each of the analysis units selected in the project, the physical space was organized and prepared so that all the required elements were available, as expressed in the physical distribution alternatives presented in Fig. 2.

In this case, the distribution in which the person conducting the study observes from a non-participatory perspective and less visible to the unit of analysis, i.e. from the back of the child with ASD, is preferred. This option has less control over the development of the tasks requested by the software and promotes independence in the child, while allowing direct observation of the reactions achieved during each prepared activity. It also highlights the realization of monitoring and collection of emotional responses from a non-invasive and limited approach to capturing facial expressions while performing the proposed activities on the mobile device.

It should highlight the location of a camera on the front of the child interacting with each mobile app, this in order to identify and document the emotional states that the child goes through during their interaction with each application used. In our case, it should have been tested on several occasions with cameras located on tripods or the use of a good quality webcam for the closest identification of facial movements generated by emotional diagnostic images in application users Computer.

With the above in mind, 2 working days were carried out with each unit of analysis, in which it was sought to observe the emotional state at the time of use of the selected Apps and the weighting of the time spent on each occasion. In these sessions a mobile device (Tablet HP Slate 7) was arranged with the selected applications installed and in front a pc with webcam activated with the e-Motion application for capturing emotions under the technique of facial recognition (Fig. 6).

Fig. 6.
figure 6

Usability assessment development

Full size image

The algorithm contained in E-motion allows to recognize the type of emotion expressed by validating the movements in the muscles of the face. In our case, the emotion expressed by autistic girls during each type of activity carried out with the inclusive apps that were selected for the case study was validated (Fig. 7).

Fig. 7.
figure 7

Facial recognition technique

Full size image

Identifying Prevailing Emotions During the Case Study.

After several experimentation sessions with autistic girls where the inclusive applications selected for this stage of the project were used in the context of pictogram management-based treatment, a relationship was obtained between the Activity performed and the predominant Emotion that this originated (See Table 5 below).

Table 5. Expressions generated in case study
Full size table

6 Results

Through this procedure, expert evaluation results are collected, in addition to their use during working sessions with the analysis units for the identification of those computational tools that best perform from the approach of usability and user-centered design.

The results obtained are the formula for calculating the percentage of usability achieved by each APP and establishing the best characteristics that an inclusive application suitable for use in the treatment of autism in the development of emotional skills.

From the point of view of the design elements of the evaluated applications, the average values of each heuristically valued aspect are analyzed leaving at the end a base of ten (10) tools that are considered to be the best designed for handling emotions in autistic children. With these identified features, a reverse engineering process is expected to obtain a framework of technical software infrastructure features suitable for the design of inclusive applications that are valid for the treatment of Autism.

The project obtains as more relevant results a list of validated computational tools for formal linking to clinical treatment processes of autism spectrum disorder; A base of user-centric design recommendations for inclusive applications is also achieved, including the design process, a gamification model, and a software architecture proposal. Finally, the project facilitated the generation of new knowledge related to the area of human-computer interaction applied in clinical contexts.

The requirements engineering models for inclusive applications for the treatment of autism add new factors to take into account, which should ensure the development of applications with a much better degree from the point of view functional, that is, from its usability and accessibility for children with ASD.

During the development of a computational application, once the functionalities that the system must cover together with the rest of its features derived from the context of the interaction are resolved, the activity design and design of the as the main activities that make up the overall process of designing such an interaction. In addition to this, the quality attributes mentioned by ISO 25010 and the gameplay characteristics required for gamified applications should be considered, therefore a cycle of evaluation (or improvement) of the application must be included develop within its architectural definition. This is where the MPIu+a models and the MDA or 6D design frameworks mentioned to ensure that functional and non-functional requirements are achieved from the start of application design and not to the end of testing with children who are immersed in ASD treatment.

To define usability criteria, the cognitive skills of children are mainly taken into account, the same as those specified in each of the development phases of the Agile Extreme Programming (XP) methodology, where each of them yields a product which is used as input for the following definition, thus generating specific and different criteria.

Software development may be framed in the use of Facade, Usability and Feedback design patterns that ensure stability and use of the final application.

The study of the characteristics (functional and non-functional) of computational tools focused on the specific use of children with autism, if allowed the determination of user-centered design alternatives for inclusive applications to support the development of emotional and social skills within the current therapeutic conditions of ASD intervention. The above validates the hypothesis raised at the beginning of the investigation.

It is also clear that the design of computer applications focused on clinical treatments such as ASD must be in line with existing quality standards. It could also be experimented with the use of gambling techniques (Gamification) that can result in a motivating factor for the interactivity between the autistic child and software applications that are used as therapeutic support.

7 Conclusions

It is possible to formulate a framework that seeks the parameterization of the design of computer applications that support the treatment of children with ASD, especially for the development of some specific predefined skills. A framework for the design of inclusive computational applications for TEA treatment can be based on the use of established software architecture patterns; however, these should be adjusted to the extent that therapeutic and technical conditions require it. The MPIu+a Model remains a guide to the engineering process of usability and accessibility par excellence, however, cases of disability it may require some methodological adjustment for the achievement of objectives desired.

The design models of computer applications, such as MDA, 6D and MPIu+a, that provide help for autism, but these have generally been used autonomously and without rigorous monitoring by the clinical specialist or without the involvement of the family of the child in the treatment follow-up.

These same models, especially MDA and 6D, facilitate the development of computer applications for contexts of application of gamified elements, but taking into account the end user for whom they must be designed, that is, children with autistic disorder. Regarding the design of these specialized software applications, there are generalized models, patterns and recommendations for inclusive software design, but these lack some particular elements for the treatment of syndromes such as ASD.

The state of the art and state of the technique finds the existence of inclusive application design models (such as MPIu+a) from which the proposal to create a specific framework based on human interaction can be initiated. computer, for the adaptation of inclusive computer applications, suitable to support the treatment of autism. This model of integration of inclusive computer applications and elements specific to the treatment of autism must be elaborated step by step and with a consequent validation of results from the most relevant usability and accessibility heuristics.

In any case, the lessons learned from the first part of the project determine that the appropriate heuristics must be defined for the evaluation of the computational tools studied and the design of workspaces adapted for children with autism. This facilitates the preparation of children with ASD, which is essential for the development of practices and tests of use of computer applications.

Likewise, the design of computer applications focused on clinical treatments such as ASD, must be adjusted to existing quality standards. Likewise, one could experiment with the use of game techniques (Gamification) that can result in a motivating factor for interactivity between the autistic child and the software applications that are used as therapeutic support.

It is possible to formulate a framework that seeks to parameterize the design of computer applications that support the treatment of children with ASD, especially for the development of some specific predefined skills. A framework for the design of inclusive computational applications for the treatment of ASD can start from the use of established software architecture patterns; however, these must be adjusted to the extent that the therapeutic and technical conditions require. The MPIu+a Model continues to be a guide for the usability and accessibility engineering process par excellence, however, cases of disability, it may require some methodological adjustment to achieve the desired particular objectives.