Multi-representation Ensembles and Delayed SGD Updates Improve Syntax-based NMT
Accepted version
Peer-reviewed
Repository URI
Repository DOI
Change log
Authors
Saunders, Danielle https://orcid.org/0000-0002-7943-3102
Stahlberg, Felix https://orcid.org/0000-0002-0430-5704
De Gispert, Adrià
Byrne, WJ
Abstract
We explore strategies for incorporating target syntax into Neural Machine Translation. We specifically focus on syntax in ensembles containing multiple sentence representations. We formulate beam search over such ensembles using WFSTs, and describe a delayed SGD update training procedure that is especially effective for long representations like linearized syntax. Our approach gives state-of-the-art performance on a difficult Japanese-English task.
Description
Keywords
machine translation, neural machine translation
Journal Title
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Conference Name
ACL 2018: 56th Annual Meeting of the Association for Computational Linguistics
Journal ISSN
Volume Title
P18-2051
Publisher
Association for Computational Linguistics
Publisher DOI
Sponsorship
EPSRC (1632937)
Engineering and Physical Sciences Research Council (EP/L027623/1)
Engineering and Physical Sciences Research Council (EP/L027623/1)
This work was supported by EPSRC grant EP/L027623/1.