Repository logo
 

Quantitative interpretation explains machine learning models for chemical reaction prediction and uncovers bias

Published version
Peer-reviewed

Change log

Authors

Kovács, Dávid Péter  ORCID logo  https://orcid.org/0000-0002-0854-2635
McCorkindale, William  ORCID logo  https://orcid.org/0000-0001-6052-8833

Abstract

Abstract: Organic synthesis remains a major challenge in drug discovery. Although a plethora of machine learning models have been proposed as solutions in the literature, they suffer from being opaque black-boxes. It is neither clear if the models are making correct predictions because they inferred the salient chemistry, nor is it clear which training data they are relying on to reach a prediction. This opaqueness hinders both model developers and users. In this paper, we quantitatively interpret the Molecular Transformer, the state-of-the-art model for reaction prediction. We develop a framework to attribute predicted reaction outcomes both to specific parts of reactants, and to reactions in the training set. Furthermore, we demonstrate how to retrieve evidence for predicted reaction outcomes, and understand counterintuitive predictions by scrutinising the data. Additionally, we identify Clever Hans predictions where the correct prediction is reached for the wrong reason due to dataset bias. We present a new debiased dataset that provides a more realistic assessment of model performance, which we propose as the new standard benchmark for comparing reaction prediction models.

Description

Keywords

Article, /639/638/549/973, /639/638/630, /639/638/563/606, /639/705/1042, article

Journal Title

Nature Communications

Conference Name

Journal ISSN

2041-1723

Volume Title

12

Publisher

Nature Publishing Group UK