Repository logo
 

Improving sample and feature selection with principal covariates regression

Published version
Peer-reviewed

Change log

Abstract

jats:titleAbstract</jats:title> jats:pSelecting the most relevant features and samples out of a large set of candidates is a task that occurs very often in the context of automated data analysis, where it improves the computational performance and often the transferability of a model. Here we focus on two popular subselection schemes applied to this end: CUR decomposition, derived from a low-rank approximation of the feature matrix, and farthest point sampling (FPS), which relies on the iterative identification of the most diverse samples and discriminating features. We modify these unsupervised approaches, incorporating a supervised component following the same spirit as the principal covariates (PCov) regression method. We show how this results in selections that perform better in supervised tasks, demonstrating with models of increasing complexity, from ridge regression to kernel ridge regression and finally feed-forward neural networks. We also present adjustments to minimise the impact of any subselection when performing unsupervised tasks. We demonstrate the significant improvements associated with PCov-CUR and PCov-FPS selections for applications to chemistry and materials science, typically reducing by a factor of two the number of features and samples required to achieve a given level of regression accuracy.</jats:p>

Description

Funder: Trinity College, University of Cambridge; doi: http://dx.doi.org/10.13039/501100000727

Keywords

46 Information and Computing Sciences, 4611 Machine Learning, Generic health relevance

Journal Title

Machine Learning: Science and Technology

Conference Name

Journal ISSN

2632-2153
2632-2153

Volume Title

2

Publisher

IOP Publishing
Sponsorship
H2020 European Research Council (677013-HBMAP)
Swiss National Supercomputing Centre (s1000, s960)