Classification and Reconstruction of High-Dimensional Signals from Low-Dimensional Features in the Presence of Side Information
Authors
Wang, L
Yuan, X
Yang, J
Reeves, G
Calderbank, R
Carin, L
Rodrigues, M
Publication Date
2016-09-07Journal Title
IEEE Transactions on Information Theory
Conference Name
2015 IEEE International Symposium on Information Theory (ISIT)
ISSN
0018-9448
Publisher
IEEE
Volume
62
Issue
11
Pages
6459-6492
Language
English
Type
Conference Object
This Version
VoR
Metadata
Show full item recordCitation
Renna, F., Wang, L., Yuan, X., Yang, J., Reeves, G., Calderbank, R., Carin, L., & et al. (2016). Classification and Reconstruction of High-Dimensional Signals from Low-Dimensional Features in the Presence of Side Information. IEEE Transactions on Information Theory, 62 (11), 6459-6492. https://doi.org/10.1109/TIT.2016.2606646
Abstract
This paper offers a characterization of fundamental limits on the classification and reconstruction of high-dimensional signals from low-dimensional features, in the presence of side information. We consider a scenario where a decoder has access both to linear features of the signal of interest and to linear features of the side information signal; while the side information may be in a compressed form, the objective is recovery or classification of the primary signal, not the side information. The signal of interest and the side information are each assumed to have (distinct) latent discrete labels; conditioned on these two labels, the signal of interest and side information are drawn from a multivariate Gaussian distribution that correlates the two. With joint probabilities on the latent labels, the overall signal-(side information) representation is defined by a Gaussian mixture model. By considering bounds to the misclassification probability associated with the recovery of the underlying signal label, and bounds to the reconstruction error associated with the recovery of the signal of interest itself, we then provide sharp sufficient and/or necessary conditions for these quantities to approach zero when the covariance matrices of the Gaussians are nearly low rank. These conditions, which are reminiscent of the well-known Slepian–Wolf and Wyner–Ziv conditions, are the function of the number of linear features extracted from signal of interest, the number of linear features extracted from the side information signal, and the geometry of these signals and their interplay. Moreover, on assuming that the signal of interest and the side information obey such an approximately low-rank model, we derive the expansions of the reconstruction error as a function of the deviation from an exactly low-rank model; such expansions also allow the identification of operational regimes, where the impact of side information on signal reconstruction is most relevant. Our framework, which offers a principled mechanism to integrate side information in high-dimensional data problems, is also tested in the context of imaging applications. In particular, we report state-of-the-art results in compressive hyperspectral imaging applications, where the accompanying side information is a conventional digital photograph.
Keywords
classification, reconstruction, Gaussian mixture models, diversity-order, MMSE, misclassification probability, side information
Sponsorship
This work was supported in part by the (PIDDAC) for Future Health/Faculdade de Engenharia da Universidade do Porto under Grant NORTE-07-0124-FEDER-000068, funded by the Fundo Europeu de Desenvolvimento Regional through the Programa Operacional do Norte, in part by the National Funds, through FCT/MEC (PIDDAC), in part by the Royal Society International Exchanges Scheme under Grant IE120996, in part by AFOSR, in part by ARO, in part by DARPA, in part by DOE, in part by NGA, and in part by ONR. F. Renna was supported by the European Union’s Horizon 2020 Research and Innovation Programme under the Marie Skłodowska-Curie under Grant 655282. M. R. D. Rodrigues was supported by EPSRC under Grant EP/K033166/1.
Funder references
European Commission (655282)
Embargo Lift Date
2100-01-01
Identifiers
External DOI: https://doi.org/10.1109/TIT.2016.2606646
This record's URL: https://www.repository.cam.ac.uk/handle/1810/263799
Rights
Attribution 4.0 International, Attribution 4.0 International, Attribution 4.0 International
Statistics
Total file downloads (since January 2020). For more information on metrics see the
IRUS guide.