Nuclear magnetic resonance (NMR) spectroscopy is a powerful tool for studying the structural and electronic properties of paramagnetic solids. However, the interpretation of paramagnetic NMR spectra is often challenging as a result of the interactions of unpaired electrons with the nuclear spins of interest. In this work, we extend the formalism of the paramagnetic NMR shielding in the presence of spin-orbit coupling towards solid systems with multiple paramagnetic centers. We demonstrate how the single-ion electron paramagnetic resonance $g$ tensor is defined and calculated in periodic paramagnetic solids. We then calculate the hyperfine tensor and the $g$ tensor with density functional theory to show the validity of the presented model and we further demonstrate how these interactions can be combined to give the overall paramagnetic shielding tensor, $\sigma$. The method is applied to a series of olivine-type Li$TM$PO$\mathrm{<mrow\; data-mjx-texclass="ORD">4</mrow>}$ materials (with $TM$=Mn, Fe, Co, and Ni) and the corresponding $\mathrm{<mrow\; data-mjx-texclass="ORD">7</mrow>}$$Li$ and $\mathrm{<mrow\; data-mjx-texclass="ORD">31</mrow>}$$P$ NMR spectra are simulated. We analyze the effects of spin-orbit coupling and of the electron-nuclear magnetic interactions on the calculated NMR parameters. A detailed comparison is presented between contact and dipolar interactions across the Li$TM$PO$\mathrm{<mrow\; data-mjx-texclass="ORD">4</mrow>}$ series, in which the magnitudes and signs of the nonrelativistic and relativistic components of the overall isotropic shift and shift anisotropy are computed and rationalized.