We explain labor productivity differences of the service sector between Europe and the U.S. through the labor allocation taking place within the service sector. We measure labor productivity using a multisector structural transformation model that decomposes services into 11 sub-sectors comparable across Europe and the U.S. We identify wholesale and retail trade as well as business services to be the two sectors responsible for most of the lack of catch-up in labor productivity between Europe and the U.S. We also investigate which institutional characteristics are associated with the different performances of sectoral productivity across sectors. We empirically explore our country-sector panel measures of labor productivity levels, and our results suggest that differences in taxation, pro-business attitudes, ICT diffusion and rates of innovation are disproportionally correlated with the productivity of wholesale, retail and business services relative to the rest of the sectors in the economy.