Empirical Bayes Estimators for High-Dimensional Sparse Vectors


Type
Article
Change log
Authors
Srinath, Pavan 
Venkataramanan, Ramji  ORCID logo  https://orcid.org/0000-0001-7915-5432
Abstract

The problem of estimating a high-dimensional sparse vector θRn from an observation in i.i.d. Gaussian noise is considered. The performance is measured using squared-error loss. An empirical Bayes shrinkage estimator, derived using a Bernoulli-Gaussian prior, is analyzed and compared with the well-known soft-thresholding estimator. We obtain concentration inequalities for the Stein's unbiased risk estimate and the loss function of both estimators. The results show that for large n, both the risk estimate and the loss function concentrate on deterministic values close to the true risk.

Depending on the underlying θ, either the proposed empirical Bayes (eBayes) estimator or soft-thresholding may have smaller loss. We consider a hybrid estimator that attempts to pick the better of the soft-thresholding estimator and the eBayes estimator by comparing their risk estimates. It is shown that: i) the loss of the hybrid estimator concentrates on the minimum of the losses of the two competing estimators, and ii) the risk of the hybrid estimator is within order 1n of the minimum of the two risks. Simulation results are provided to support the theoretical results. Finally, we use the eBayes and hybrid estimators as denoisers in the approximate message passing (AMP) algorithm for compressed sensing, and show that their performance is superior to the soft-thresholding denoiser in a wide range of settings.

Description
Keywords
Journal Title
Information and Inference: A Journal of the IMA
Conference Name
Journal ISSN
2049-8764
2049-8772
Volume Title
9
Publisher
Oxford University Press
Sponsorship
European Commission (631489)
Engineering and Physical Sciences Research Council (EP/N013999/1)
Isaac Newton Trust (1540 (R))
This work was supported in part by a Marie Curie Career Integration Grant (Grant Agreement Number 631489), an Isaac Newton Trust Research Grant, and EPSRC Grant EP/N013999/1.