Repository logo
 

Research data supporting “Source Sentence Simplification for Statistical Machine Translation”


No Thumbnail Available

Type

Dataset

Change log

Authors

Hasler, Eva 
de, Gispert Adrià 
Stahlberg, Felix 
Waite, Aurelien 
Byrne, Bill 

Description

This data set contains subsets of English-German test sets from the Workshop for Machine Translation (WMT) which have been annotated with manual text simplification information on the source side in the form of gap begin and gap end symbols (, ). The data was tokenized and truecased using the processing scripts distributed with the Moses SMT system. The source simplifications were produced by workers recruited on the crowdsourcing platform Crowdflower (https://www.crowdflower.com). We asked workers to simplify a sentence by deleting words and punctuation, while trying to retain the most important information in the shortened sentence. Their performance was controlled using test questions and a second Crowdflower task which asked workers to identify bad simplifications from the first task. The outcomes of the second task were aggregated by combining an agreement score and the average worker trust score for each simplification. We selected randomly from the remaining simplifications with a combined score of at least 0.5.

Version

Software / Usage instructions

text editor

Keywords

text simplification, machine translation

Publisher

Sponsorship
Engineering and Physical Sciences Research Council (EP/L027623/1)
EPSRC [EP/L027623/1]
Relationships
Supplements: