Show simple item record

dc.contributor.authorAhfock, DC
dc.contributor.authorAstle, WJ
dc.contributor.authorRichardson, S
dc.date.accessioned2022-06-21T23:31:31Z
dc.date.available2022-06-21T23:31:31Z
dc.date.issued2021-06
dc.identifier.issn0006-3444
dc.identifier.urihttps://www.repository.cam.ac.uk/handle/1810/338289
dc.description.abstractSketching is a probabilistic data compression technique that has been largely developed by the computer science community. Numerical operations on big datasets can be intolerably slow; sketching algorithms address this issue by generating a smaller surrogate dataset. Typically, inference proceeds on the compressed dataset. Sketching algorithms generally use random projections to compress the original dataset, and this stochastic generation process makes them amenable to statistical analysis. We argue that the sketched data can be modelled as a random sample, thus placing this family of data compression methods firmly within an inferential framework. In particular, we focus on the Gaussian, Hadamard and Clarkson-Woodruff sketches and their use in single-pass sketching algorithms for linear regression with huge samples. We explore the statistical properties of sketched regression algorithms and derive new distributional results for a large class of sketching estimators. A key result is a conditional central limit theorem for data-oblivious sketches. An important finding is that the best choice of sketching algorithm in terms of mean squared error is related to the signal-to-noise ratio in the source dataset. Finally, we demonstrate the theory and the limits of its applicability on two datasets.
dc.description.sponsorshipThis research was conducted using the UK Biobank resource. Richardson was supported by the UKRI Medical Research Council and the Alan Turing Institute. Astle was supported by NHS Blood and Transplant and the National Institute for Health Research Blood and Transplant Research Unit.
dc.format.mediumPrint-Electronic
dc.publisherOxford University Press (OUP)
dc.rightsAttribution 4.0 International
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectComputational efficiency
dc.subjectRandom projection
dc.subjectRandomized numerical linear algebra
dc.subjectSketching
dc.titleStatistical properties of sketching algorithms.
dc.typeArticle
dc.publisher.departmentMrc Biostatistics Unit
dc.date.updated2022-06-21T12:57:00Z
prism.endingPage297
prism.issueIdentifier2
prism.publicationDate2021
prism.publicationNameBiometrika
prism.startingPage283
prism.volume108
dc.identifier.doi10.17863/CAM.85697
dcterms.dateAccepted2020-05-27
rioxxterms.versionofrecord10.1093/biomet/asaa062
rioxxterms.versionVoR
dc.contributor.orcidAstle, William [0000-0001-8866-6672]
dc.contributor.orcidRichardson, Sylvia [0000-0003-1998-492X]
dc.identifier.eissn1464-3510
rioxxterms.typeJournal Article/Review
pubs.funder-project-idMRC (Unknown)
cam.issuedOnline2020-07-30
cam.depositDate2022-06-21
pubs.licence-identifierapollo-deposit-licence-2-1
pubs.licence-display-nameApollo Repository Deposit Licence Agreement


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record

Attribution 4.0 International
Except where otherwise noted, this item's licence is described as Attribution 4.0 International