Repository logo

Neural Grammatical Error Correction with Finite State Transducers

Accepted version


Conference Object

Change log


Bryant, Christopher 
Byrne, Bill 


Grammatical error correction (GEC) is one of the areas in natural language processing in which purely neural models have not yet superseded more traditional symbolic models. Hybrid systems combining phrase-based statistical machine translation (SMT) and neural sequence models are currently among the most effective approaches to GEC. However, both SMT and neural sequence-to-sequence models require large amounts of annotated data. Language model based GEC (LM-GEC) is a promising alternative which does not rely on annotated training data. We show how to improve LM-GEC by applying modelling techniques based on finite state transducers. We report further gains by rescoring with neural language models. We show that our methods developed for LM-GEC can also be used with SMT systems if annotated training data is available. Our best system outperforms the best published result on the CoNLL-2014 test set, and achieves far better relative improvements over the SMT baselines than previous hybrid systems.



cs.CL, cs.CL

Journal Title

Conference Name

2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)

Journal ISSN

Volume Title



All rights reserved
EPSRC (1632937)
EPSRC (1632937)
EPSRC grant EP/L027623/1