Repository logo
 

Span-ConveRT: Few-shot span extraction for dialog with pretrained conversational representations

Accepted version
Peer-reviewed

Type

Conference Object

Change log

Authors

Coope, S 
Farghly, T 
Gerz, D 
Vulić, I 
Henderson, M 

Abstract

We introduce Span-ConveRT, a light-weight model for dialog slot-filling which frames the task as a turn-based span extraction task. This formulation allows for a simple integration of conversational knowledge coded in large pretrained conversational models such as ConveRT (Henderson et al., 2019). We show that leveraging such knowledge in Span-ConveRT is especially useful for few-shot learning scenarios: we report consistent gains over 1) a span extractor that trains representations from scratch in the target domain, and 2) a BERT-based span extractor. In order to inspire more work on span extraction for the slot-filling task, we also release RESTAURANTS-8K, a new challenging data set of 8,198 utterances, compiled from actual conversations in the restaurant booking domain.

Description

Keywords

Journal Title

Proceedings of the Annual Meeting of the Association for Computational Linguistics

Conference Name

ACL 2020: 58th Annual Meeting of the Association for Computational Linguistics

Journal ISSN

0736-587X

Volume Title

Publisher

Association for Computational Linguistics

Rights

All rights reserved
Sponsorship
European Research Council (648909)