Repository logo

Automatic Assessment of Conversational Speaking Tests

Accepted version


Conference Object

Change log


McKnight, Simon Webster 
Civelekoglu, Arda 
Gales, Mark JF 
Banno, Stefano 
Liusie, Adian 


Many speaking tests are conversational, dialogic, in form with an interlocutor talking to one or more candidates. This paper investigates how to automatically assess such a test. State-of-the-art approaches are used in a multi-stage pipeline: diarization and speaker assignment, to detect who is speaking and when; automatic speech recognition (ASR), to produce a transcript; and finally assessment. Each presents challenges which are investigated in the paper. Advanced foundation model-based auto-markers are examined: an ensemble of Longformer-based models that operates on the ASR output text; and a wav2vec2-based system that works directly on the audio. The two are combined to yield the final score. This fully automated system is evaluated in terms of ASR performance, and related impact of candidate assignment, as well as prediction of the candidate mark on data from the Occupational English Test. This is a conversational speaking test for L2 English healthcare professionals.



Journal Title

Conference Name

Workshop on Speech and Language Technology in Education (SLaTE)

Journal ISSN

Volume Title


Publisher DOI

Publisher URL

Cambridge Assessment (unknown)
Cambridge Assessment (Unknown)
This paper reports on research supported by Cambridge University Press \& Assessment (CUP&A), a department of The Chancellor, Masters, and Scholars of the University of Cambridge. The OET data was provided by Cambridge Boxhill Language Assessment Trust (CBLA).