Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval

Conference Object
Change log
Litschko, R 
Vulić, I 
Ponzetto, SP 
Glavaš, G 

Pretrained multilingual text encoders based on neural Transformer architectures, such as multilingual BERT (mBERT) and XLM, have achieved strong performance on a myriad of language understanding tasks. Consequently, they have been adopted as a go-to paradigm for multilingual and cross-lingual representation learning and transfer, rendering cross-lingual word embeddings (CLWEs) effectively obsolete. However, questions remain to which extent this finding generalizes 1) to unsupervised settings and 2) for ad-hoc cross-lingual IR (CLIR) tasks. Therefore, in this work we present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks across a large number of language pairs. In contrast to supervised language understanding, our results indicate that for unsupervised document-level CLIR -- a setup with no relevance judgments for IR-specific fine-tuning -- pretrained encoders fail to significantly outperform models based on CLWEs. For sentence-level CLIR, we demonstrate that state-of-the-art performance can be achieved. However, the peak performance is not met using the general-purpose multilingual text encoders "off-the-shelf", but rather relying on their variants that have been further specialized for sentence understanding tasks

46 Information and Computing Sciences
Journal Title
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Conference Name
Proceedings of the 43rd European Conference on Information Retrieval (ECIR 2021)
Journal ISSN
Volume Title
12656 LNCS
Springer International Publishing
All rights reserved
European Research Council (648909)