ISCA Archive - Overview of the 2017 Spoken CALL Shared Task
ISCA Archive SLaTE 2017
ISCA Archive SLaTE 2017

Overview of the 2017 Spoken CALL Shared Task

Claudia Baur, Cathy Chua, Johanna Gerlach, Manny Rayner, Martin Russell, Helmer Strik, Xizi Wei

We present an overview of a first shared task for spoken CALL. Groups competed on a prompt-response task using English-language data collected, using an online CALL game, from Swiss German teens in their second and third years of learning English. Each item consists of a written German prompt and an audio file containing a spoken reply. The task is to accept linguistically correct responses and reject linguistically incorrect ones, to match a gold standard; scoring was performed using a metric defined as the ratio of the relative rejection rates on incorrect and correct responses. The task received twenty entries from nine different groups. We present the task itself, the results, a tentative analysis of what makes items challenging, correlations between different possible metrics, and suggestions for a continuation.