A search with minimal model dependence for physics beyond the Standard Model in events featuring three or four leptons ($3\ell$ and $4\ell$, $\ell =e,\mu$) is presented. The analysis aims to be sensitive to a wide range of potential new-physics theories simultaneously. This analysis uses data from $pp$ collisions delivered by the Large Hadron Collider at a centre-of-mass energy of $s=13$ TeV and recorded with the ATLAS detector, corresponding to the full Run 2 dataset of 139 fb$\mathrm{<mrow\; data-mjx-texclass="ORD">-1</mrow>}$. The $3\ell$ and $4\ell$ phase space is divided into 22 event categories according to the number of leptons in the event, the missing transverse momentum, the invariant mass of the leptons, and the presence of leptons originating from a $Z$-boson candidate. These event categories are analysed independently for the presence of deviations from the Standard Model. No statistically significant deviations from the Standard Model predictions are observed. Upper limits for all signal regions are reported in terms of the visible cross-section.