Abstractive spoken document summarization using hierarchical model with multi-stage attention diversity optimization

Conference Object
Change log
Gales, MJF 
Wang, L 

Abstractive summarization is a standard task for written documents, such as news articles. Applying summarization schemes to spoken documents is more challenging, especially in situations involving human interactions, such as meetings. Here, utterances tend not to form complete sentences and sometimes contain little information. Moreover, speech disfluencies will be present as well as recognition errors for automated systems. For current attention-based sequence-to-sequence summarization systems, these additional challenges can yield a poor attention distribution over the spoken document words and utterances, impacting performance. In this work, we propose a multi-stage method based on a hierarchical encoder-decoder model to explicitly model utterance-level attention distribution at training time; and enforce diversity at inference time using a unigram diversity term. Furthermore, multitask learning tasks including dialogue act classification and extractive summarization are incorporated. The performance of the system is evaluated on the AMI meeting corpus. The inclusion of both training and inference diversity terms improves performance, outperforming current state-of-the-art systems in terms of ROUGE scores. Additionally, the impact of ASR errors, as well as performance on the multitask learning tasks, is evaluated.

abstractive spoken document summarization, hierarchical model, attention diversity, multitask learning
Journal Title
Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
Conference Name
Interspeech 2020
Journal ISSN
Volume Title
All rights reserved
Cambridge Assessment (Unknown)