The surface of a polar liquid presents a special environment for the solvation and organization of charged solutes, which differ from bulk behaviors in important ways. These differences have motivated many attempts to understand electrostatic response at aqueous interfaces in terms of a spatially varying dielectric permittivity, typically concluding that the dielectric constant of interfacial water is significantly lower than in the bulk liquid. Such analyses, however, are complicated by the potentially nonlocal nature of dielectric response over the short length scales of interfacial heterogeneity. Here we circumvent this problem for thin water films by adopting a thermodynamic approach. Using molecular simulations, we calculate the solvent's contribution to the reversible work of charging a parallel plate capacitor. We find good agreement with a simple dielectric continuum model that assumes bulk dielectric permittivity all the way up to the liquid's boundary, even for very thin (∼1 nm) films. This comparison requires careful attention to the placement of dielectric boundaries between liquid and vapor, which also resolves apparent discrepancies with dielectric imaging experiments.