We consider a rapidly rotating, Boussinesq fluid stirred by buoyant anomalies. In such a system it is known that, in the absence of a magnetic field, inertial waves whose wave vectors lie normal to the rotation axis play a key role in establishing quasi-geostrophic motion. In particular, buoyant anomalies radiate low-frequency inertial wave packets which disperse along the rotation axis, leading to axially elongated columnar vortices. Here we focus on the influence of an ambient magnetic field on this process, motivated by the dynamics of planetary cores. We find that, once again, the waves responsible for establishing quasi-geostrophic structures have wave vectors normal to the rotation axis; however, these are not conventional inertial waves, but rather hybrid 'inertial-Alfvén waves'. Their frequency equals that of an Alfvén wave but their axial group velocity is half that of the equivalent inertial wave. They have maximal kinetic, magnetic and cross-helicity, carry magnetic and kinetic energy in equal amounts, and are particularly potent in establishing columnar, helical vortices through the spontaneous emission of axially elongated wave packets. Although our hybrid inertial-Alfvén waves have been overlooked in dynamo literature to date, we speculate that they in fact play a central role in planetary dynamos.