Crystal nucleation is one of the most fundamental processes in the physical sciences and almost always occurs heterogeneously with the aid of a nucleating substrate. No example of nucleation is more ubiquitous and impactful than the formation of ice, vital to fields as diverse as geology, biology, aeronautics, and climate science. However, despite considerable effort, we still cannot predict a priori the efficacy of a nucleating agent. Here we utilize deep learning methods to accurately predict nucleation ability from images of room temperature liquid water-generated from molecular dynamics simulations-on a broad range of substrates. The resulting model, named IcePic, can rapidly and accurately infer nucleation ability, eliminating the requirement for either notoriously expensive simulations or direct experimental measurement. In an online poll, IcePic was found to significantly outperform humans in predicting the ice nucleating efficacy of materials. By analyzing the typical errors made by humans, as well as the application of reverse interpretation methods, physical insights into the role the water contact layer plays in ice nucleation have been obtained. Moving forward, we suggest that IcePic can be used as an easy, cheap, and rapid way to discern the nucleation ability of substrates, also with potential for learning other properties related to interfacial water.