The dust content of normal galaxies and the dust mass density (DMD) at high-z (z>4) are unconstrained given the source confusion and the sensitivity limitations of previous observations. The ALMA Large Program to INvestigate [CII] at Early Times (ALPINE), which targeted 118 UV-selected star-forming galaxies at 4.4<z<5.9, provides a new opportunity to tackle this issue for the first time with a statistically robust dataset. We have exploited the rest-frame far-infrared (FIR) fluxes of the 23 continuum individually detected galaxies and stacks of continuum images to measure the dust content of the 118 UV-selected ALPINE galaxies. We have focused on the dust scaling relations and, by comparing them with predictions from chemical evolution models, we have probed the evolutionary stage of UV-selected galaxies at high-z. By using the observed correlation between the UV-luminosity and the dust mass, we have estimated the DMD of UV-selected galaxies at z~5, weighting the galaxies by means of the UV-luminosity function (UVLF). The derived DMD has been compared with the value we have estimated from the 10 ALPINE far-IR continuum blindly detected galaxies at the redshift of the ALPINE targets. The comparison of the observed dust scaling relations with chemical evolution models suggests that ALPINE galaxies are not likely progenitors of disc galaxies, but of intermediate and low mass proto-spheroids, resulting in present-day bulges of spiral or elliptical galaxies. Interestingly, this conclusion is in line with the independent morphological analysis, that shows that the majority (~70%) of the dust-continuum detected galaxies have a disturbed morphology. The DMD obtained at z~5 from UV-selected sources is ~30% of the value obtained from blind far-IR selected sources, showing that the UV-selection misses the most dust-rich, UV-obscured galaxies.