Repository logo
 

Automatically Grading Learners’ English Using a Gaussian Process


Change log

Authors

van Dalen, RC 
Knill, KM 
Gales, MJF 

Abstract

There is a high demand around the world for the learning of English as a second language. Correspondingly, there is a need to assess the proficiency level of learners both during their studies and for formal qualifications. A number of automatic methods have been proposed to help meet this demand with varying degrees of success. This paper considers the automatic assessment of spoken English proficiency, which is still a challenging problem. In this scenario, the grader should be able to accurately assess the learner’s ability level from spontaneous, prompted, speech, independent of L1 language and the quality of the audio recording. Automatic graders are potentially more consistent than humans. However, the validity of the predicted grade varies. This paper proposes an automatic grader based on a Gaussian process. The advantage of using a Gaussian process is that as well as predicting a grade, it provides a measure of the uncertainty of its prediction. The uncertainty measure is sufficiently accurate to decide which automatic grades should be re-graded by humans. It can also be used to determine which candidates are hard to grade for humans and therefore need expert grading. Performance of the automatic grader is shown to be close to human graders on real candidate entries. Interpolation of human and GP grades further boosts performance.

Description

Keywords

spoken language assessment, Bayesian methods, Gaussian process

Journal Title

Speech and Language Technology in Education, SLaTE 2015

Conference Name

Speech and Language Technology in Education (SLaTE 2015)

Journal ISSN

Volume Title

Publisher

ISCA
Sponsorship
Cambridge Assessment (unknown)
This work was supported by Cambridge English, University of Cambridge.