Ultraluminous infrared galaxies (ULIRGs) are characterised by extreme starburst (SB) and AGN activity, and are therefore ideal laboratories for studying the outflow phenomena. We have recently started a project called Physics of ULIRGs with MUSE and ALMA (PUMA), which is a survey of 25 nearby (z < 0.165) ULIRGs observed with the integral field spectrograph MUSE and the interferometer ALMA. This sample includes systems with both AGN and SB nuclear activity in the pre- and post-coalescence phases of major mergers. The main goals of the project are to study the prevalence of multi-phase outflows as a function of the galaxy properties, to constrain the driving mechanisms of the outflows (e.g. distinguish between SB and AGN winds), and to identify feedback effects on the host galaxy. In this first paper, we present details on the sample selection, MUSE observations, and derive first data products. MUSE data were analysed to study the dynamical status of each of the 21 ULIRGs observed so far, taking the stellar kinematics and the morphological properties inferred from MUSE narrow-band images into account. We also located the ULIRG nuclei, using near-IR (HST) and mm (ALMA) data, and studied their optical spectra to infer the ionisation state through BPT diagnostics, and outflows in both ionised and neutral gas. We show that the morphological and stellar kinematic classifications are consistent: post-coalescence systems are more likely associated with ordered motions, while interacting (binary) systems are dominated by non-ordered and streaming motions. We also find broad and asymmetric [OIII] and NaID profiles in almost all nuclear spectra, with line widths in the range 300-2000 km/s, possibly associated with AGN- and SB-driven winds. This result reinforces previous findings that indicated that outflows are ubiquitous during the pre- and post-coalescence phases of major mergers.