Repository logo
 

Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints

Published version
Peer-reviewed

Type

Article

Change log

Authors

Mrkšić, Nikola 
Vulić, Ivan 
Ó Séaghdha, Diarmuid 
Leviant, Ira 
Reichart, Roi 

Abstract

We present Attract-Repel, an algorithm for improving the semantic quality of word vectors by injecting constraints extracted from lexical resources. Attract-Repel facilitates the use of constraints from mono- and cross-lingual resources, yielding semantically specialised cross-lingual vector spaces. Our evaluation shows that the method can make use of existing cross-lingual lexicons to construct high-quality vector spaces for a plethora of different languages, facilitating semantic transfer from high- to lower-resource ones. The effectiveness of our approach is demonstrated with state-of-the-art results on semantic similarity datasets in six languages. We next show that Attract-Repel-specialised vectors boost performance in the downstream task of dialogue state tracking (DST) across multiple languages. Finally, we show that cross-lingual vector spaces produced by our algorithm facilitate the training of multilingual DST models, which brings further performance improvements.

Description

Keywords

Journal Title

Transactions of the Association for Computational Linguistics (TACL)

Conference Name

Journal ISSN

2307-387X

Volume Title

5

Publisher

Association for Computational Linguistics

Publisher DOI

Sponsorship
European Research Council (648909)
Ivan Vulic, Roi Reichart and Anna Korhonen are supported by the ERC Consolidator Grant LEXICAL (number 648909). Roi Reichart is also supported by the Intel-ICRI grant: Hybrid Models for Minimally Supervised Information Extraction from Conversations.