Assessing the neural computations supporting predictions of form and meaning in speech comprehension

Abstract

Spoken language is complex: listeners must keep track of linguistic information at multiple levels of linguistic representation (e.g., phonology, semantics) in parallel as the signal unfolds over time. In addition, this input is often ambiguous: in lexicosemantic ambiguities, multiple meanings could map on to identical word forms (“bank” meaning riverbank or financial institution). In order to disambiguate and interpret ambiguities in the signal, sufficient contextual information is required. The processing of predictable words in context has been shown to be facilitated during language comprehension. This has led to the proposal that listeners predict upcoming language input at multiple levels, from higher order semantic and message levels down to phonological content. But how do listeners use predictions when interpreting spoken language? This thesis addresses two key aims. The first aim was to study linguistic prediction, specifically how predictability influences word comprehension, and whether this necessarily entails pre-activation. The second aim was to investigate the neural computational mechanisms that combine predictions (priors) with sensory input during spoken sentence processing. Chapter 1 provides a general introduction and overview of the behavioural, cognitive, neural, and computational research underlying the theoretical motivation for the analysis approach: multivariate analyses of electrophysiological human brain recordings using simultaneous magnetoencephalography and electroencephalography (M/EEG). Chapter 2 covers an additional crucial aspect of study design: the trade-off between “naturalness” and “control” in cognitive science. The following three chapters (3, 4, and 5) report the results of empirical analyses, in which the pre-activation and computational mechanisms underlying linguistic prediction were investigated using representational similarity analysis (RSA). Semantically congruent words in context elicited reduced amplitude responses compared to anomalous words, and predictable ambiguous words were reduced compared to their congruent but less predictable synonyms (Chapter 4). This result emphasizes the role of context and predictability in the comprehension of spoken words. RSA revealed neural representations of heard phonemes (Chapter 3) and the contextually predictable meanings of ambiguous words and their synonyms (Chapter 5). This is in line with sharpening computations – in which the predictable information in the stimulus is represented. While effects in favour of prediction error computations – in which the unpredictable or surprising information in the stimulus is represented – were limited, uncorrected clusters in the RSA time series revealed representations of the alternative meanings of ambiguous words (Chapter 5). Overall, the work in this thesis highlights the computation of meaning from predictable input in spoken language comprehension. These results are discussed in the context of the existing literature on predictive coding and Bayesian inference.