The inverse problem of estimating the spatial distributions of elastic material properties from noisy strain measurements is ill-posed. However, it is still typically treated as an optimisation problem to maximise a likelihood function that measures the agreement between the measured and theoretically predicted strains. Here we propose an alternative approach employing Bayesian inference with Nested Sampling used to explore parameter space and compute Bayesian evidence. This approach not only aids in identifying the basis function set (referred to here as a model) that best describes the spatial material property distribution but also allows us to estimate the uncertainty in the predictions. Increasingly complex models with more parameters generate very high likelihood solutions and thus are favoured by a maximum likelihood approach. However, these models give poor predictions of the material property distributions with a large associated uncertainty as they overfit the noisy data. On the other hand, the Bayes’ factor peaks for a relatively simple model and indicates that this model is most appropriate even though its likelihood is comparatively low. Intriguingly, even for the appropriate model that has a unique maximum likelihood solution, the measurement noise is amplified to give large errors in the predictions of the maximum likelihood solution. By contrast, the mean of the posterior probability distribution reduces the effect of noise in the data and predicts the material properties with significantly higher fidelity. Simpler model selection criteria such as the Bayesian information criterion are shown to fail due to the non-Gaussian nature of the posterior distribution of the parameters. This makes accurate evaluation of the posterior distribution and the associated Bayesian evidence integral (by Nested Sampling or other means) imperative for this class of problems. The output of the Nested Sampling algorithm is also used to construct likelihood landscapes. These landscapes show the existence of multiple likelihood maxima when there is paucity of data and/or for overly complex models. They thus graphically illustrate the pitfalls in using optimisation methods to search for maximum likelihood solutions in such inverse problems.