We use variational methods to study a spin impurity in a 1D Bose lattice gas. Both in the strongly interacting superfluid regime and the Mott regime we find that the impurity binds with a hole, forming a polaron. Our calculations for the dispersion of the polaron are consistent with recent experiments by Fukuhara et. al. [Nature Phys. 9, 235 (2013)] and give a better understanding of their numerical simulations. We find that for sufficiently weak interactions there are ranges of momentum for which the polaron is unstable. We propose experimentally studying the stability of the polaron by measuring the correlation between the impurity and holes. We also study two interacting impurities, finding stable bipolarons for sufficiently strong interactions.