Single-shot images are the standard readout of experiments with ultracold atoms, the imperfect reflection of their many-body physics. The efficient extraction of observables from single-shot images is thus crucial. Here we demonstrate how artificial neural networks can optimize this extraction. In contrast to standard averaging approaches, machine learning allows both one- and two-particle densities to be accurately obtained from a drastically reduced number of single-shot images. Quantum fluctuations and correlations are directly harnessed to obtain physical observables for bosons in a tilted double-well potential at an extreme accuracy. Strikingly, machine learning also enables a reliable extraction of momentum-space observables from real-space single-shot images and vice versa. With this technique, the reconfiguration of the experimental setup between in situ and time-of-flight imaging is required only once to obtain training data, thus potentially granting an outstanding reduction in resources.