We introduce a hybridizable discontinuous Galerkin method for the incompressible Navier--Stokes equations for which the approximate velocity field is pointwise divergence-free. The method builds on the method presented by Labeur and Wells [SIAM J. Sci. Comput., vol. 34 (2012), pp. A889--A913]. We show that with modifications of the function spaces in the method of Labeur and Wells it is possible to formulate a simple method with pointwise divergence-free velocity fields which is momentum conserving, energy stable, and pressure-robust. Theoretical results are supported by two- and three-dimensional numerical examples and for different orders of polynomial approximation.