SemEval-2020 Task 2: Predicting Multilingual and Cross-Lingual (Graded) Lexical Entailment

Abstract

Lexical entailment (LE) is a fundamental asymmetric lexico-semantic relation, supporting the hierarchies in lexical resources (e.g., WordNet, ConceptNet) and applications like natural language inference and taxonomy induction. Multilingual and cross-lingual NLP applications warrant models for LE detection that go beyond language boundaries. As part of SemEval 2020, we carried out a shared task (Task 2) on multilingual and cross-lingual LE. The shared task spans three dimensions: (1) monolingual LE in multiple languages versus cross-lingual LE, (2) binary versus graded LE, and (3) a set of 6 diverse languages (and 15 corresponding language pairs). We offered two different evaluation tracks: (a) distributional (Dist): for unsupervised, fully distributional models that capture LE solely on the basis of unannotated corpora, and (b)Any: for externally informed models, allowed to leverage any resources, including lexico-semantic networks (e.g.,WordNet or BabelNet). In the Any track, we received system runs that push state-of-the-art across all languages and language pairs, for both binary LE detection and graded LE prediction.