Towards automatic interpretation of A Fortiori arguments

Abstract

Argument A Fortiori represents a common type of everyday reasoning where the conclusions about the possibility of a certain scenario are drawn based on its comparison to some similar scenario, as in the following mini-argument:

“Implementing this law? It has not even been passed yet.”

To infer the main claim of this argument, namely that the law is not implemented yet, the human brain is able to perform a likelihood comparison, which involves comparing the probability of passing a law with the likelihood of passing a law and then implementing it. Being able to understand the underlying reasons behind A Fortiori arguments is an important step towards general argument interpretation and knowledge acquisition.

In this thesis, I present a model of the theoretical aspects behind the A Fortiori reasoning pattern that extends Sion’s (2013) model. My model explains A Fortiori via an underlying relation type, an implicit property, and the correlation between these. The model also delineates the true cases of A Fortiori against zeugmas and plain wrong A Fortiori usage. In order to validate my model, I conducted a human annotation study, using a dataset of real-world “let alone” sentences across several domains and registers. The pairwise agreement ranged from K = .42 to K = .70 for the task of relation classification and from ko = .34 to ko = .71 for the task of the hidden property specification, whereas for the task of identifying the two compared scenarios, the agreement rose to F1=.83 and F1=.89 (entity-level, strict).

I then present an automation for the three main steps of my model, using a combination of deep learning approaches with multi-task learning and transformer architectures for masked language modelling. The steps of my pipeline represent the following tasks: sequence labelling for argument component identification (where the best model achieves F1macro =.84±.012), classification for the comparison reason (where the best model achieves F1macro =.64±.005), and prediction of the hidden comparison property. The last task is automated in two ways, as a ranking task (where the best model achieves MAP=.34±.009) and as a masked word prediction task (where the best model achieves an accuracy of 66%). My results show that it is in principle possible to automate the major steps of the task of understanding A Fortiori logic. They also show that having access to the output of previous steps in the pipeline improves the performance on subsequent ones.