Repository logo

Overview of the 2019 Spoken CALL Shared Task

Accepted version


Conference Object

Change log


Baur, C 
Chua, C 
Gerlach, J 
Qian, M 


We present an overview of the third edition of the Spoken CALL Shared Task. Groups competed on a prompt-response task using English-language data collected, through an online CALL game, from Swiss German teens in their second and third years of learning English. Each item consists of a written German prompt and an audio file containing a spoken response. The task is to accept linguistically correct responses and reject linguistically incorrect ones, with “linguistically correct” defined by a gold standard derived from human annotations. Scoring was performed using a metric based on the idea of maximising the ratios correct-accept-rate/false-accept-rate and correct-reject-rate/false-reject-rate. The third edition received sixteen entries, with the best score substantially improving on last year's edition of the task. We analyse factors which make it difficult to label items correctly, concluding that, as in the previous edition, good speech recognition is most important. Finally, we suggest a strategy for continuing the task.



Journal Title

8th ISCA Workshop on Speech and Language Technology in Education, SLaTE 19

Conference Name

8th ISCA Workshop on Speech and Language Technology in Education (SLaTE 2019)

Journal ISSN

Volume Title