Compressive sensing of images and video: towards low-complexity, real-time operation
University of Cambridge
Doctor of Philosophy (PhD)
MetadataShow full item record
Zammit, J. (2020). Compressive sensing of images and video: towards low-complexity, real-time operation (Doctoral thesis). https://doi.org/10.17863/CAM.69798
Compressive Sensing (CS) acquires sparse signals with far fewer measurements than samples required by the classical Nyquist sampling theorem, at the cost of more computationally intensive reconstruction. Video Block Compressive Sensing (VBCS), using a Multi Pixel Camera (MPC), divides the sensed image into blocks reducing storage requirements, and allowing lower latency capture in parallel at the sensor. These approaches suffer from low performance or long reconstruction times. Distributed Video Coding (DVC) techniques emerged at the turn of the millennium for compressing video for transmission from resource constrained devices. The rationale behind this thesis is to combine VBCS and DVC to develop a video transmission system having a low power implementation. The focus is on three areas: the development of real-time, computationally efficient sensor-side CS video frame compression and decoder-side reconstruction techniques, the quantization and coding of image and video measurements for transmission, and the exploitation of temporal correlation between frames to improve performance. With respect to the compression of independent video frames, this work proposes two novel classes of techniques: Adaptive Block Compressive Sensing (ABCS) and re-purposed single image super resolution. In the first class, two novel algorithms are proposed that adapt the number of measurements in each block using initial measurements in the spatial or discrete cosine transform domains. These achieve State of the Art (SOTA) performance amongst ABCS algorithms. A SOTA Iterative Denoising Algorithm (IDA) is also developed that can deblock and improve the rate-distortion quality, with a time penalty. In the second class, an algorithm (Shrink) is developed that combines a bicubic interpolator with an antialiasing filter to perform downsampling on an MPC. GPU-based reconstruction algorithms are used to super resolve the downsampled images at the decoder in real time improving quality at high compression. Two algorithms are developed to quantize and code ABCS sensed frames for digital transmission: a JPEG-like algorithm and a novel, low-complexity table based algorithm: TAB. At low compression factors, the best performance is achieved by the JPEG-like algorithm that allows the ABCS techniques to match the JPEG rate-distortion performance even though the frames are compressively sensed. TAB performs better at higher compression factors. Shrink is encoded with a low-complexity algorithm based on Golomb coding and achieves better rate-distortion than the end-to-end ABCS algorithms at high compression. Temporal correlation is exploited at the encoder to better adapt the number of measurements per block. At the decoder, it increases the quality of the highly compressed non-key frames to match that of key frames that are periodically coded with higher quality. ABCS techniques are found to exploit temporal correlation significantly better than Shrink based techniques. A proposed VBCS algorithm using differential pulse code modulation and video frame interpolation is shown to achieve SOTA results from a dimensionality reduction perspective. Finally, the TAB algorithm is improved to better compress encoder measurements for end-to-end transmission.
This record's DOI: https://doi.org/10.17863/CAM.69798
All rights reserved