The manner in which electrolyte solutions respond to electric fields is crucial to understanding the behavior of these systems both at, and away from, equilibrium. The present formulation of linear response theory for such systems is inconsistent with common molecular dynamics (MD) implementations. Using the finite field formalism, suitably adapted for finite temperature MD, we investigate the response of bulk aqueous NaCl solutions to both finite Maxwell ($E$) and electric displacement ($D$) fields. The constant $E$ Hamiltonian allows us to derive the linear response relation for the ionic conductivity in a simple manner that is consistent with the forces used in conventional MD simulations. Simulations of a simple point charge model of an electrolyte solution at constant $E$ yield conductivities at infinite dilution within 15% of experimental values. The finite field approach also allows us to measure the solvent's dielectric constant from its polarization response, which is seen to decrease with increasing ionic strength. Comparison of the dielectric constant measured from polarization response versus polarization fluctuations enables direct evaluation of the dynamic contribution to this dielectric decrement, which we find to be small but not insignificant. Using the constant $D$ formulation, we also rederive the Stillinger-Lovett conditions, which place strict constraints on the coupling between solvent and ionic polarization fluctuations.