Methods known as Lipschitz Interpolation or Nonlinear Set Membership regression have become established tools for nonparametric system-identification and data-based control. They utilise presupposed Lipschitz properties to compute inferences over unobserved function values. Unfortunately, it relies on the a priori knowledge of a Lipschitz constant of the underlying target function which serves as a hyperparameter. We propose a closed-form estimator of the Lipschitz constant that is robust to bounded observational noise in the data. The merger of Lipschitz Interpolation with the new hyperparameter estimator gives a new nonparametric machine learning method for which we derive sample complexity bounds and online learning convergence guarantees. Furthermore, we apply our learning method to model-reference adaptive control. We provide convergence guarantees on the closed-loop dynamics and compare the performance of our approach to recently proposed alternative learning-based controllers in a simulated flight manoeuvre control scenario.