We present dust attenuation properties of spectroscopically confirmed star forming galaxies on the main sequence at redshift ~4.4-5.8. Our analyses are based on the far infrared continuum observations of 118 galaxies at rest-frame $158\mu m$ obtained with the ALMA large program ALPINE. We study the connection between the UV spectral slope ($\beta$), stellar mass ($M\star$), and infrared excess (IRX$=LIR/LUV$). Twenty-three galaxies are individually detected in the continuum at $>3.5\sigma$ significance. We perform a stacking analysis using both detections and non-detections to study the average dust attenuation properties at z~4.4-5.8. The individual detections and stacks show that the IRX-$\beta$ relation at z~5 is consistent with a steeper dust attenuation curve than typically found at lower redshifts (z<4). The attenuation curve is similar to or even steeper than that of the extinction curve of the Small Magellanic Cloud (SMC). This systematic change of the IRX-$\beta$ relation as a function of redshift suggests an evolution of dust attenuation properties at $z>4$. Similarly, we find that our galaxies have lower IRX values up to 1 dex on average at fixed mass compared to previously studied IRX-$M\star$ relations at $z\lesssim 4$, albeit with significant scatter. This implies a lower obscured fraction of star-formation than at lower redshifts. Our results suggest that dust properties of UV-selected star forming galaxies at $z\gtrsim 4$ are characterised by (i) a steeper attenuation curve than at $z\lesssim 4$, and (ii) a rapidly decreasing dust obscured fraction of star formation as a function of redshift. Nevertheless, even among this UV-selected sample, massive galaxies ($\log M\star /M\odot >10$) at z~5-6 already exhibit an obscured fraction of star formation of $\sim 45\%$, indicating a rapid build-up of dust during the epoch of reionization.