Perovskite photovoltaic (PV) cells have demonstrated power conversion efficiencies (PCE) that are close to those of monocrystalline silicon cells; however, in contrast to silicon PV, perovskites are not limited by Auger recombination under 1-sun illumination. Nevertheless, compared to GaAs and monocrystalline silicon PV, perovskite cells have significantly lower fill factors due to a combination of resistive and non-radiative recombination losses. This necessitates a deeper understanding of the underlying loss mechanisms and in particular the ideality factor of the cell. By measuring the intensity dependence of the external open-circuit voltage and the internal quasi-Fermi level splitting (QFLS), the transport resistance-free efficiency of the complete cell as well as the efficiency potential of any neat perovskite film with or without attached transport layers are quantified. Moreover, intensity-dependent QFLS measurements on different perovskite compositions allows for disentangling of the impact of the interfaces and the perovskite surface on the non-radiative fill factor and open-circuit voltage loss. It is found that potassium-passivated triple cation perovskite films stand out by their exceptionally high implied PCEs > 28%, which could be achieved with ideal transport layers. Finally, strategies are presented to reduce both the ideality factor and transport losses to push the efficiency to the thermodynamic limit.