Just(ifying) algorithms: Data-driven automated predictions about unobservable targets and the General Data Protection Regulation

Abstract

Predictive algorithms are increasingly used to profile, rank, classify, and make decisions about groups and individuals. Data-driven machine learning techniques, in which statistical models are developed and updated without any initial prior theory about which data are related to a classification, and how, now account for many applications. Despite increasing demand for data-driven prediction, there is broad academic, policy and societal consensus that these techniques create various and novel risks of harm. However, it remains unclear whether data protection law, specifically the General Data Protection Regulation (GDPR), requires both the process and outcomes of data-driven predictions to be substantively justified. While some scholars have argued that the GDPR does not contain a ‘right to reasonable inferences’, recent case law and regulatory guidance suggest that the data protection principles in Article 5 of the GDPR, read with related provisions, may be more relevant than previously suggested. This thesis examines whether, and in what ways, individuals are protected against substantively unjustified predictions about their future behaviour under the GDPR. It first extends the discourse on algorithmic fairness by delineating inherent technical limitations of data-driven prediction. Distinguishing between targets of interest which are unobservable, evaluative and merely unobserved, the thesis identifies a number of irreducibly normative questions about the first of these. Drawing on these insights, the thesis then engages in a close analysis of case law and regulatory guidance on the interpretation of relevant provisions of the GDPR, especially the previously underexplored data protection principles. The thesis argues that judicial and regulatory interpretations of the GDPR are evolving as datadriven prediction and its underlying technologies place increasing pressure on data protection law to protect individuals against unfair predictions about their future behaviour. The data protection principles have normative force of their own, and are increasingly and consistently interpreted as operating independently of other provisions. Relevant provisions require substantive as well as procedural fairness of inputs and outputs. As such, this thesis clarifies the relevance, nature and application of normative standards for prediction under the GDPR. The unrelenting development and deployment of data-driven predictive models suggests that the trend towards broader interpretation of the GDPR will continue.