The most promising avenue for detecting primordial gravitational waves from cosmic inflation is through measurements of degree-scale CMB B-mode polarisation. This approach must face the challenge posed by gravitational lensing of the CMB, which obscures the signal of interest. Fortunately, the lensing effects can be partially removed by combining high-resolution E-mode measurements with an estimate of the projected matter distribution. For near-future experiments, the best estimate of the latter will arise from co-adding internal reconstructions (derived from the CMB itself) with external tracers such as the cosmic infrared background (CIB). In this work, we characterise how foregrounds impact the delensing procedure when CIB intensity, I, is used as the matter tracer. We find that higher-point functions of the CIB and Galactic dust such as ⟨BEI⟩c and ⟨EIEI⟩c can, in principle, bias the power spectrum of delensed B-modes. To quantify these, we first estimate the dust residuals in currently-available CIB maps and upcoming, foreground-cleaned Simons Observatory CMB data. Then, using non-Gaussian simulations of Galactic dust -- extrapolated to the relevant frequencies, assuming the spectral index of polarised dust emission to be fixed at the value determined by Planck -- we show that the bias to any primordial signal is small compared to statistical errors for ground-based experiments, but might be significant for space-based experiments probing very large angular scales. However, mitigation techniques based on multi-frequency cleaning appear to be very effective. We also show, by means of an analytic model, that the bias arising from the higher-point functions of the CIB itself ought to be negligible.