Surfaces of ionic solids interacting with an ionic solution can build up charge by exchange of ions. The surface charge is compensated by a strip of excess charge at the border of the electrolyte forming an electric double layer. These electric double layers are very hard to model using the supercell's methods of computational condensed phase science. The problem arises when the solid is an electric insulator (as most ionic solids are) permitting a finite interior electric field over the width of the slab representing the solid in the supercell. The slab acts as a capacitor. The stored charge is a deficit in the solution failing to compensate fully for the solid surface charge. Here, we show how these problems can be overcome using the finite field methods developed by Stengel, Spaldin, and Vanderbilt [Nat. Phys. 5, 304 (2009)]. We also show how the capacitance of the double layer can be computed once overall electric neutrality of the double layer is restored by application of a finite macroscopic field $E$ or alternatively by zero electric displacement $D$. The method is validated for a classical model of a solid-electrolyte interface using the finite-temperature molecular dynamics adaptation of the constant field method presented previously [C. Zhang and M. Sprik, Phys. Rev. B 93, 144201 (2016)]. Because ions in electrolytes can diffuse across supercell boundaries, this application turns out to be a critical illustration of the multivaluedness of polarization in periodic systems.