Keywords

1 Introduction

Over the past two decades, there has been much interest in the use of interactive media as learning tools. Due to their interactive, entertaining and engaging nature, they have been successfully employed as learning aids for children with cognitive impairments. They have been applied in teaching curriculum subjects such as mathematics [1], as well and independent living skills [2] with positive results. More recently though, research on the use of interactive media for learning has been moving towards creating rather than consuming pre-designed content [3]. One of the main forces driving this movement is the availability of easy to use creative software, among which are Visual Programming (VP) tools.

A VP language “allows the user to specify a program in a two (or more) dimensional fashion” [4]. VP tools integrate a VP language and an environment that supports the visualization of program execution. Recently, several visual programming tools have been developed to improve children’s access to programming skills at an early age; they include Scratch [5] and Alice [6].

Scratch allows users to create interactive and media rich projects such as animated stories, games, and music videos. It is free and open source, and can be used either online, or through its offline desktop application [5]. Scratch projects are made up of media (2D backdrops and sprites, and sounds) and scripts. Media can be built, imported or created within Scratch, and scripts are made up of blocks of instructions. The main aim of Scratch is “to introduce programming to those with no previous programming experience” [7]. Hence its features such as the use of visual blocks, single-window user interface layout with multiple panes, minimal command set, and its no error approach to error handling. It focuses less on direct instructions and encourages learning through exploration. Therefore, Scratch provides a sandbox environment with few rules and no restrictions as to what one can create (Fig. 1).

Fig. 1.
figure 1

A screenshot of a Scratch project created on the Scratch website.

Full size image

VP tools like Scratch can potentially play an important role in learning and teaching for those with cognitive impairments. They can provide a platform for fostering creativity, improving communication skills, building new interests, learning computational and problem-solving skills, and ultimately lead to inclusion. However, conducted literature review showed little is being done to utilize them as learning aids for children with cognitive impairments. Furthermore, very little is known about their accessibility to individuals that face the difficulties associated with cognitive impairments.

This paper presents a study conducted with the aim of evaluating the accessibility of Scratch for children with cognitive impairments. With the objectives of identifying the difficulties they face, and the effects of those difficulties. The section that follows discusses the research methodology used, the participants, and the process of data collection and analysis. Findings of the study are presented in detail in the third section, followed by a section that reflects on the findings and their implications. Conclusion and future work are presented in the fifth and final section.

2 Methodology

The aim of this study is to evaluate the accessibility of Scratch for children with cognitive impairments. Due to the nature of the participants and the research goal, a Grounded Theory [8] approach was followed. Grounded Theory “consists of a systematic yet flexible guidelines for collecting and analysing qualitative data to construct a theory from the data themselves” [9]. It is best suited for situations where the goal is to understand, from the perspective of people, how they understand and deal with challenging situations. It does not require a prior hypothesis to focus research on [10], hence it can be applied in situations where very little is known about the issue to be researched.

Although this method has its origins in social sciences, it has been adopted by researchers outside the field, including HCI researchers [11,12,13].

2.1 Participants

Six special education schools catering for individuals with different needs and age groups were contacted within the greater Nottinghamshire area, and one agreed to participate in the study. Given the nature of the population with regards to diversity and unique needs, the school was consulted in choosing participants for the study. A criterion for inclusion was provided to the school; participants should be less than 18 years old, are able to and have experience using a web browser on a computer and have been diagnosed as being cognitively impaired.

A class which consisted of 8 students was recommended by the school as the group that’s stands to benefit the most from a study of this kind. Out of the eight members of the class, seven participated in the study and one was disqualified because of his inability to use a web browser. Participants were aged between thirteen and fourteen years old and comprised of four females and three males. Table 1 below shows the participants’ profiles provided by the class teacher.

Table 1. Participants’ diagnosed cognitive impairment and fine motor skills.
Full size table

Two of the participating children have been diagnosed with learning disability only. Learning disability affects an individual’s ability to understand information, communicate, and learn new things (both academic and non-academic). It can also affect one’s attention, memory, coordination, social skills, and emotional maturity [14]. Its nature, i.e. moderate or severe, determines how adverse the difficulties are on the individual.

The remaining five participating children have been diagnosed with both learning disability and autism spectrum disorder. Autism spectrum disorders (ASD) refer to a range of conditions that result in difficulties interacting appropriately with others, possessing language and communication disorders and peculiarities, struggling with changes in the environment, and peculiar repetitive behaviour [15]. However, being a spectrum disorder, individuals with ASD tend to have unique needs and requirements.

Two class teachers (T1 and T2), one male and one female, also participated in the study. Between them, they had an average of more than eleven years of experience in teaching children with special needs.

2.2 Data Collection and Analysis

Even though the most common sources of data used in grounded theory are collected through interviews and observations, almost any form of written, observed and recorded data can be used [16]. This study employed a combination of video screen capture and semi-structured interviews.

Screen capture is performed using software that “records the actions, sounds and movements that take place on electronic monitors” [17]. A free and open source screen capturing software (Camstudio [18]) was used for screen capturing in this study. It was used to collect empirical data on the interactions between the participants and Scratch.

Participants were introduced to Scratch and its features in an introductory session that lasted for one hour. In each of the ten weeks that followed, at least two of the seven participating children used Scratch to create stories during a 45-min storytelling session. Storytelling was chosen by the class teachers because of its potential in improving the participants’ communication skills. Graphical guides for accomplishing common tasks (based on previous stories participants have created) were designed and provided to participants. The guides consisted of scripts for participants to duplicate, and instructions written in communication symbols. The researcher acted as a participant observer, and at least one teaching assistant was available to support participants per session. All sessions were held in the participants’ classroom, using their classroom laptops, and were screen captured using Camstudio. To tackle this methods limitation of capturing only onscreen data [19], audio data of participants interacting with their environment was included in the recordings.

As recommended in Grounded Theory, data collection and analysis were conducted simultaneously. The process usually starts with transcribing data, then coding. Coding is the process of assigning words or phrases that capture or summarize the concepts present within a portion of language based or visual data [20]. The grounded theory coding method defined by Corbin and Strauss [16] was used in this study. It has three stages: open, axial and selective coding. Concepts are identified from data and allocated codes during open coding. Related concepts are then combined to form categories. Axial coding is conducted next, this phase involves further developing categories of data and finding relationships that exist between categories and sub categories [21], using the “coding paradigm” of conditions, context, strategies and consequences [16]. The final coding phase is selective coding, it involves the unification of all categories around a central category; the central phenomenon of the study. A descriptive narrative, which has the central category at its centre is then developed [22]. Constant comparison should be done until additional data adds no new knowledge about the concepts and categories generated i.e. saturation has been reached.

Data analysis in this study was done using QSR Nvivo 11 [23] data analysis software. All video screen capture data was watched three times, first to transcribe interesting observations, then to code them as concepts, and lastly to identify and verify relationships between concepts. Concepts were combined into sub categories and categories. As the study concerned accessibility, all accessibility issues identified in the data were scrutinized to determine their causal conditions, the context in which they occurred, the strategies adopted by participants in the situation, and their consequences. However, saturation was reached before a theory was fully formed.

Based on the emerging theory from the screen capture data analysed, semi-structured interviews were conducted with the two class teachers. In Grounded Theory, where focus is on data collection for the sake of creating a theory, semi-structured interviews are valuable tools for researchers to target concepts that they consider relevant for their theory. Collected data was analysed in the same way the screen capture data was, and used to verify and complete the theory.

During this process more than 70 concepts were identified. After comparing and verifying data with data, a theory was developed with 5 main categories, 20 sub-categories and 69 concepts.

3 Findings

Following an iterative and rigorous process of coding, comparing codes, and verification with the data itself, concepts were categorized into phenomena, causal conditions, context, strategies and consequences. Phenomena represents the study’s main interest, which is accessibility issues in this case. Accessibility issues observed in this study were classified into two types, user interface (UI) related and cognition related issues. The study’s findings lead to the creation of a theoretical model of how these accessibility issues affect the use of scratch by children with cognitive impairments.

In the sub-sections that follow, each category is discussed in detail. Examples of observations from the screen capture videos as well as quotations established from the interview transcripts are used to support interesting discussions. Although both types of accessibility issues belong to one category, they have been presented separately for clarity. A table showing a category’s subcategories (if any), concepts, and an example of a concept’s occurrence follows each category discussion. Observations and quotations include one or more of the following individuals: a participant (P1–7), the researcher conducting the session, a teaching assistant (TA) and a class teacher (T1–2). Finally, the theoretical model is presented.

3.1 UI Related Issues

These accessibility issues are directly related to how the Scratch user interface is designed, and how it is interacted with. They are discussed below, and summarised in Table 2.

Table 2. UI related issues, with observation examples for selected concepts.
Full size table

Recognising Buttons.

Findings from this study revealed that at one point or another all participants struggled to recognise buttons on the user interface. Buttons for importing media such as sprites and backdrops caused difficulties at the beginning of sessions but seemed to be easier to identify with time. However, participants continuously struggled with differentiating buttons for shrink, grow and delete tools. In one scenario observed, to shrink the size of a knight sprite, P2 repeatedly chose and used the grow tool, which did not produce the intended results.

Differentiating Links.

Links in Scratch can be used to access collections of similar items. For example, when importing backdrops or sprites, links ease the process of locating items by grouping images into themes or categories. This study observed that participants rarely utilised this feature while working alone. Participants had to be advised to use the feature to locate specific items. For example, in one instance, P7 was trying to add a dragon sprite to his story, a TA advised P7 to look in ‘Fantasy’. P7 then clicked on two wrong links, before the TA pointed him to the right link.

Another use for links in Scratch is to access block categories. When instructing participants, it was observed that they were only able to identify block category links when they were associated with a colour (block categories are colour coded). Simply using the text associated with the category’s link was not enough. For example, in one session, P2 asked for help on integrating a sound into his story, he was instructed to click to the ‘Sound’ block category, he then hovered over ‘Looks’ for a few moments until the link was associated with a colour. However, it was observed that although all categories are coded with unique colours, some of the colours are similar. As a result, some categories were difficult to identify even when associated with colours.

Differentiating Blocks.

Scratch uses a unique colour to identify each category of blocks, which means all the blocks within a category have the same colour. Although a few blocks are labelled with both text and a symbol, most blocks’ unique identifier is their text label. Observations showed participants in this study struggled to differentiate blocks of the same category. This was recorded in both cases where participants were following instructions, and when participants were creating their own scripts.

Switching Area.

Most Scratch projects require users to move continuously between the scripts, sounds and costumes area using tabs. However, some participants kept finding themselves stuck in one area and unable to switch to another. For example, when P7 customised a sprite in the costumes area and wanted to get back to the scripts area, he attempted that by trying to minimise the browser, when that didn’t work he clicked the browsers ‘Back’ button, that brought up a ‘Save’ message box and the attention of a TA who showed P7 how to switch areas using tabs.

Other observations showed the struggle of participants who understood the purpose of the tabs but could not tell one from the other. Those participants used trial and error to find the area they were looking for.

Dragging.

Participants, especially those with fine motor skills difficulties struggled with dragging items within the Scratch interface. They had difficulties positioning their sprites when setting up their stories, as well as dragging blocks to create scripts. However, when it came to rearranging blocks in a script, observations showed all participants faced difficulties in dragging a block to a new position.

Selecting.

To use tools such as grow, shrink, delete and tools within Scratch’s paint editor, one must first click on the required tool to select it. Participants with poor fine motor skills also found this task difficult. Especially when the tool is small and/or closely positioned to another tool. Another group of items of that were difficult to select were sprite thumbnails for switching active sprite or choosing an active sprite for a costume. In these situations, participants were observed to be dragging slightly, when their intention was to click.

3.2 Cognition Related Issues

These accessibility issues are related to the cognitive processes required to efficiently use Scratch. They are discussed below, and summarised in Table 3.

Table 3. Cognition related issues, with observation examples for selected concepts.
Full size table

Defining Instruction.

In Scratch, each component of a project that has a behaviour needs to have that role explicitly defined using scripts. On many occasions observed, participants expected their sprites to perform some form of action with no script defined. Participants that faced this difficulty were observed to execute their projects without adding any script again and again, or expect a script assigned to one sprite to also work on another just by activating the second sprite.

Structuring and Sequencing.

All participants, at one point or another struggled with structuring and sequencing the events in their stories. When participants were creating stories in the moment, without any initial plans, they seemed to lack the direction and guidance they needed to develop ideas into their projects, and easily went back to recreating old storylines from previous sessions.

When participants had a predetermined story to create, they faced other kinds of difficulties. Such as struggling to correctly sequence the actions of sprites, and on fewer occasions, struggling to reinitialise the positions of sprites after execution.

Staying on Track.

This study observed that participants were easily distracted from the goal of creating a story. This happened usually because of a discovery within the software in form of media, a feature of the software, or a new skill. The discovery then led to loosing focus on the overall story to focus on one aspect or abandoning the story altogether. For example, when P5 was introduced to Scratch’s paint editor, she spent the next 13 min modifying her sprites.

However, not all recorded instances of participants getting distracted were due to distractions from within Scratch, distractions from the participants’ surroundings have also been recorded. For example, during one session P6 was in the middle of creating a story when he overhead a conversation about choosing a Scratch username for P7, P6 then abandoned his story to add sprites related to the username mentioned.

3.3 Causal Conditions

These conditions lead to the occurrences of accessibility issues. They are discussed below, and summarised in Table 4.

Table 4. Causal conditions of accessibility issues with observation examples for selected concepts.
Full size table

Text Labels as Identifiers.

Throughout this study, participants struggled with aspects of Scratch that required reading and writing skills. Links and blocks that are identified using text labels were a constant source of struggle for participants. Their writing and spelling difficulties were also evident when they attempted to input or change block inputs, e.g. when inputting the text to be displayed by a ‘Say’ block.

Similar Colours in Proximity.

Although using colours helped participants recognise and differentiate objects, using similar colours to differentiate objects within proximity defeated the purpose. This was mostly observed when participants were trying to locate a block category coded with a colour that’s unique, yet similar to that of other categories. A possible reason for this confusion may be found in the interview excerpt below:

“…. even without visual impairment, visual processing is quite a difficult thing for them.” (T1)

Lack of Templates.

Although Scratch provides a guide for creating example projects, it does not provide a structure or template for creating a project. This led to participants having difficulty in developing their story ideas or structuring them in the right way within Scratch. T1 mentions how participants can be helped in that aspect using concrete objects:

“… that’s where students struggle, having that imagination. By having props, that’s sort of your concrete, real bit” (T1)

Templates or interactive guides could have been applied as a concrete frame in the scenario, for participants to build projects around.

Lack of Constraints.

One of the most appealing features of Scratch is that it has no restrictions on what users can create. But for the participants in this study, this might be a disadvantage due to the difficulty they have staying focused. T1 explains why having focus is important, and how it’s implemented in the classroom, below:

“We do that already with our curriculum planning don’t we, because if we give ourselves a title that we want, and then that sort of immediately gives you a bit of focus, a bit of clarity… for example, if you are doing a story, and you said ‘Right, we are doing something about the sea’, then obviously that gets rid of a load, and that’s what you are trying to do. Otherwise there is far too much choice out there. But in saying that, what we try and do here is get it to be student led, so what are you interested in, talk to the students about it” (T1)

Lack of constraints did not just affect participants that found it difficult to stay focused on tasks, but also those participants that were repetitive in performing tasks, choosing characters, story themes, etc. T2 discusses how the lack of constraint affected P7 and how restrictions may help, below:

“he can’t move on, like his favourite colour is red and he has to paint everything red… if he was working with the same 10 set of characters, what would happen if he uses the program and those 10 characters weren’t there, he’d have to use something else.” (T2)

With Scratch’s sandbox nature, users with characteristics similar to participants of this study are likely to get side-tracked from their original goals.

Mouse Input.

Participants, especially those with poor fine motor skills struggled with mouse operations such as dragging, double clicking and even clicking. Clicking was especially difficult on objects that can be clicked as well as dragged. T1 explains why one participant struggles with mouse operations in the excerpt below:

“…she would find it extremely difficult just to pick something up off the table, because she doesn’t have that depth in perception. If you are using a mouse, for it to register your finger, you can’t be sort of up and down.” (T1)

Scratch being a drag and drop environment, relies highly on mouse input, hence caused a lot of difficulties to this group of participants.

3.4 Context

It was observed that different usability issues occurred in different contexts of usage, they are discussed below, and summarised in Table 5.

Table 5. Context for the occurrences of accessibility issues with observation examples of selected concepts.
Full size table

Adding and Modifying Media.

Identified UI related issues mostly occurred during two aspects of using Scratch, the first is when participants were working with media e.g. sprites and backdrops. At this stage participants were interested in locating, adding or positioning, and may choose to customize imported media to fit their needs. The mentioned operations required identifying links, buttons, tools, selecting, and may also require dragging.

Scripting.

The second aspect of Scratch that triggered UI related difficulties is scripting. In creating scripts for their stories, participants had to select the right script, locate the right block category, identify the right block and drag it to the scripting area.

Planning and Implementing Story.

Cognition related accessibility issues affected participants trying to breakdown their story ideas into components that can be represented within Scratch. This involved building up a setting, characters and storylines. For instance, structuring and sequencing difficulties were mostly observed when participants added sprites and were trying to decide how best to organize and add scripts to them. Difficulties with staying focused on the other hand were mostly recorded when participants were integrating their interests in the story such as specific characters or themes, or when they decided to implement changes to their stories.

3.5 Strategies

Observations showed that participants used one of four strategies when they faced difficulties. The strategies are discussed below, and summarised in Table 6.

Table 6. Strategies employed when accessibility difficulties are faced with examples of selected concepts.
Full size table

Idle.

This strategy was more common among participants with communication difficulties. When facing an issue, or when they were not sure how to proceed after completing a task, these participants normally became idle. For example, during a session where P1 was creating a story about a dragon and pony, she became idle after adding the pony sprite and its corresponding script, until she was noticed by the researcher.

Ask for Help.

Other participants reacted to difficulties by asking for help from either the TA or the researcher. Participants asked for help in performing tasks ranging from navigating the user interface to implementing a storyline. For instance, when P2 was trying to locate a diver to include in his story, he asked the researcher who advised him to look under the ‘People’ category.

Retry.

Retrying was another strategy used by participants to tackle difficulties, mainly UI related ones. Observations showed how participants attempted clicking or dragging multiple times until they achieved their objective, tried out multiple tools until the right one was chosen, had a look at different links to find the right group of media etc.

Move to Another Task.

The last strategy that was observed is moving to another task. In some cases, it was employed as a second strategy, after being idle for a while or retrying with no success. Participants then moved to a different task which may be related or unrelated to their current project.

3.6 Consequences

Consequences show the actions taken by the researcher or TA to address accessibility issues, or a strategy. They are discussed below, and summarised in Table 7.

Table 7. Consequences of facing accessibility issues or employing strategy, with observation examples for selected concepts.
Full size table

Participant is Kept on Track.

When a TA or the researcher noticed a participant that is idle, taking too much time on a single task, or performing tasks not related to their story, they tried to get the participant back on track and provide help where necessary. This was done by drawing the participant’s attention to other aspects of the story using questions or discussion. For example, when P5 was caught up in editing her character, she was simply asked questions about the characters role in the story, and P5 got interested again in creating the story not just the character. This was noticed to be an efficient way of reminding participants of other aspects of the project that they needed to work on.

Participant is Instructed.

The researcher or TA provided guidance to participants that asked a question or were seen to be idle. The process usually started by the researcher or TA confirming what the participant wanted to achieve, and then providing an explanation that can help achieve that goal, if that failed then step by step guidance was provided on how to achieve the goal.

Task is Performed for Participant.

Lastly, in cases where the participant tried multiple times alone, or under instruction to achieve a goal unsuccessfully, the researcher or TA performed the task while the participant watched. This was the approach that was taken during a session where after P4 unsuccessfully attempted to rearrange blocks in a script.

3.7 Theoretical Model

Based on the findings presented in Sects. 3.1–3.6, the theoretical model shown below was generated to explain the experiences of participants in this study while using Scratch to create stories (Fig. 2).

Fig. 2.
figure 2

A theoretical model showing how accessibility issues affect the use of Scratch.

Full size image

4 Discussions

This study used a Grounded Theory approach to evaluate the accessibility of Scratch, and how that affects its usage by children with cognitive impairments. Findings uncovered two types of accessibility issues faced by participants, those that were caused by the design of the UI and how it was interacted with, and those that had to do with the cognitive processes of using Scratch.

It was discovered that participants had trouble locating and identifying buttons, links and blocks. Although Scratch uses colours to differentiate objects, not all objects are uniquely identified by colours. In cases where colours are unique it was found that similar colours within proximity easily confused participants. Although few buttons and blocks are identified with text and symbols, text labels are the main unique identifiers used by Scratch. They were difficult for participants to recognise due to reading difficulties and led to most of the difficulties identifying objects on the UI. Difficulties of this kind usually occurred when participants were working with media (adding, modifying) or blocks.

Even though the difficulties associated with the UI outnumber those related to cognition, the latter was more difficult to tackle and had greater impact on participants’ goal of creating stories. Scratch is well known for the freedom it provides its users to create projects, while that can be considered good for creativity, it’s not necessarily helpful to this study’s participants. Due to this lack of restriction, some participants frequently strayed off track after discovering new media objects or new features. Other participants on the other hand chose to focus on a task or keep repeating an action. With no template to build around, participants also found it difficult to define instructions and behaviour, structure projects and sequence actions. These difficulties resulted in non-goal oriented projects, distractions, and creative difficulties.

Participants employed strategies when faced by difficulties depending on the participant’s characteristics, and difficulty. Participants either became idle, retried, asked for help or moved to another task. Staying on track was the only difficulty that did not lead to any strategy by the participant and was only identified by either the researcher or a TA, who then tried to get the participant back on track.

Consequences occurred as a result of a participant facing a difficulty or employing a strategy. They ranged from simply keeping the participant on track, providing an explanation for a participant, instructing the participant in simple steps on how to perform a task, to, if all else failed, the researcher or teaching assistant performing the task for the participant.

5 Conclusion

The aim of this study was to evaluate the accessibility of Scratch for children with cognitive impairments. The study used a novel approach to accessibility evaluation by employing a Grounded Theory research method. The resulting findings make original contributions to knowledge on the accessibility of visual programming tools, and shed light on the experiences of users with cognitive impairments. Not only are the accessibility issues reported, their causes, practices of users when faced with difficulties and measures taken to help users are also reported upon. This provides guidance on best practices while using such tools in similar contexts, but more importantly it provides accessible design suggestions for visual programming environments suitable for those with cognitive impairments.

It should be acknowledged that this study has its limitations and they are as follows. First, extensive programming was not taught to participants before or during the study and participants had no previous programming experiences, rather participants were guided to create simple programming scripts that they required to create stories of the complexity that they normally created before using Scratch. Therefore, programming difficulties were not considered, but difficulties related to following programming guides were included. Secondly, even though teaching assistants provided guidance and help to participants during sessions, they were not themselves trained on how to use Scratch, which meant they were also required to learn and experiment with the software as part of the sessions. However, despite this limitation, there were few instances where a teaching assistant was unsure on how to guide participants, and in those cases the researcher was asked for support. Lastly, it should be noted that the participants in this study represent only a small group of individuals with cognitive impairments, which means that all needs and requirements of children with cognitive impairments while using Scratch may not have been addressed in this study.

Future work should focus on investigating how specific cognitive impairments, for example Autism Spectrum Disorders, affect the accessibility of Scratch as a storytelling tool. By focusing on one type of impairment, specific characteristics, needs and requirements that need to be considered to make Scratch accessible can be better understood.