Repository logo
 

SemEval-2020 Task 3: Graded Word Similarity in Context

Published version
Peer-reviewed

Type

Conference Object

Change log

Authors

Santos Armendariz, Carlos 
Purver, Matthew 
Pollak, Senja 
Ljubesic, Nikola 
Ulcar, Matej 

Abstract

This paper presents the Graded Word Similarity in Context (GWSC) task which asked participants to predict the effects of context on human perception of similarity in English, Croatian, Slovene and Finnish. We received 15 submissions and 11 system description papers. A new dataset (CoSimLex) was created for evaluation in this task: it contains pairs of words, each annotated within two short text passages. Systems beat the baselines by significant margins, but few did well in more than one language or subtask. Almost every system employed a Transformer model, but with many variations in the details: WordNet sense embeddings, translation of contexts, TF-IDF weightings, and the automatic creation of datasets for fine-tuning were all used to good effect.

Description

Keywords

Journal Title

Proceedings of the 14th International Workshop on Semantic Evaluation (SemEval 2020)

Conference Name

14th International Workshop on Semantic Evaluation (SemEval 2020)

Journal ISSN

Volume Title

Publisher

International Committee for Computational Linguistics
Sponsorship
European Research Council (648909)