We introduce a framework for the reconstruction of the amplitude, phase and polarisation of an optical vector-field using measurements acquired by an imaging device characterised by an integral transform with an unknown spatially-variant kernel. By incorporating effective regularisation terms, this new approach is able to recover an optical vector-field with respect to an arbitrary representation system, which may be different from the one used for device calibration. In particular, it enables the recovery of an optical vector-field with respect to a Fourier basis, which is shown to yield indicative features of increased scattering associated with tissue abnormalities. We demonstrate the effectiveness of our approach using synthetic holographic images as well as biological tissue samples in an experimental setting where measurements of an optical vector-field are acquired by a multicore fibre (MCF) endoscope, and observe that indeed the recovered Fourier coefficients are useful in distinguishing healthy tissues from tumours in early stages of oesophageal cancer.