A search for Higgs boson pair production in the $bb¯WW\ast$ decay mode is performed in the $bb¯l\nu qq$ final state using 36.1 fb$\mathrm{<mrow\; data-mjx-texclass="ORD">-1</mrow>}$ of proton-proton collision data at a centre-of-mass energy of 13 TeV recorded with the ATLAS detector at the Large Hadron Collider. No evidence of events beyond the background expectation is found. Upper limits on the non-resonant $pp\to HH$ production cross section of 10 pb and on the resonant production cross section as a function of the $HH$ invariant mass are obtained. Resonant production limits are set for scalar and spin-2 graviton hypotheses in the mass range 500 to 3000 GeV.