Repository logo
 

Missing the missing values: The ugly duckling of fairness in machine learning

Published version
Peer-reviewed

Change log

Authors

Fernando, Martínez‐Plumed  ORCID logo  https://orcid.org/0000-0003-2902-6477
José, Hernández‐Orallo  ORCID logo  https://orcid.org/0000-0001-9746-7632

Abstract

Abstract: Nowadays, there is an increasing concern in machine learning about the causes underlying unfair decision making, that is, algorithmic decisions discriminating some groups over others, especially with groups that are defined over protected attributes, such as gender, race and nationality. Missing values are one frequent manifestation of all these latent causes: protected groups are more reluctant to give information that could be used against them, sensitive information for some groups can be erased by human operators, or data acquisition may simply be less complete and systematic for minority groups. However, most recent techniques, libraries and experimental results dealing with fairness in machine learning have simply ignored missing data. In this paper, we present the first comprehensive analysis of the relation between missing values and algorithmic fairness for machine learning: (1) we analyse the sources of missing data and bias, mapping the common causes, (2) we find that rows containing missing values are usually fairer than the rest, which should discourage the consideration of missing values as the uncomfortable ugly data that different techniques and libraries for handling algorithmic bias get rid of at the first occasion, (3) we study the trade‐off between performance and fairness when the rows with missing values are used (either because the technique deals with them directly or by imputation methods), and (4) we show that the sensitivity of six different machine‐learning techniques to missing values is usually low, which reinforces the view that the rows with missing data contribute more to fairness through the other, nonmissing, attributes. We end the paper with a series of recommended procedures about what to do with missing data when aiming for fair decision making.

Description

Keywords

46 Information and Computing Sciences, 4611 Machine Learning, Machine Learning and Artificial Intelligence, Networking and Information Technology R&D (NITRD)

Journal Title

International Journal of Intelligent Systems

Conference Name

Journal ISSN

0884-8173
1098-111X

Volume Title

36

Publisher

Hindawi Limited
Sponsorship
Ministerio de Economía, Industria y Competitividad, Gobierno de España (ES) (RTI2018‐094403‐B‐C3)
Generalitat Valenciana (PROMETEO/2019/09)
Future of Life Institute (RFP2‐15)
European Commission (DG JRC ‐ HUMAINT project)