Quantum phase transitions (QPT) on the border of magnetism have provided a fertile hunting ground for the discovery of new states of matter, for example; the marginal Fermi Liquid and non Fermi Liquid states as well high T cuprate and magnetically mediated superconductivity. In this thesis I present work on three materials in which it may be possible to tune the system through a magnetic QPT with the application of hydrostatic pressure. Although the details of the underlying physics are different in each of the materials, they are linked by the possibility of finding new states on the border of magnetism. Applying hydrostatic pressure, we have suppressed the ferromagnetic (FM) transition in metallic Fe$2$P to very low temperature and to a potential QPT. Counter-intuitive broadening of the magnetic hysteresis leading up to the FM-AFM QPT may well be a crucial clue as to the nature of the model needed to understand this phase transition. A sharp increase in the quasi-particle scattering cross-section as well as the residual resistivity accompany a departure from the quadratic temperature dependence of the resistivity. This possible deviation from Fermi liquid behaviour is stable over a significant range of temperature. The unexplained upturn in the resistivity of CeGe that accompanies the AFM transition was studied under pressure. Pressure increased the residual resistivity as well as decreasing the relative size of the upturn, but had a moderate effect on the Neel temperature. The insensitivity of the N$e´$el temperature to pressure has been compared to its relative sensitivity to applied feld. The existence of the upturn and its evolution with pressure and applied feld can reasonably be argued to be due to the details of the electron band structure in the system. By applying pressure we have drastically reduced the resistivity of the insulating antiferromagnet NiPS$3$. Concurrent work on FePS$3$ has shown metallisation under pressure. It seems reasonable to speculate that NiPS$3$ may also metallise at higher pressure. The energy gap is narrowed in both materials as pressure is increased. Magnetisation measurements have revealed a low temperature upturn indicating some possible ferromagnetic component or proximity to another magnetic state. A peak in the magnetisation is also seen at 45K in zero-feld cooled measurements. Both of these features point to a system with a complex magnetic ground state.