SL(2,Z) duality transformations in asymptotically AdS4 x S^7 act non-trivially on the three-dimensional SCFT of coincident M2-branes on the boundary. We show how S-duality acts away from the IR fixed point. We develop a systematic method to holographically obtain the deformations of the boundary CFT and show how electric-magnetic duality relates different deformations. We analyze in detail marginal deformations and deformations by dimension 4 operators. In the case of massive deformations, the RG flow relates S-dual CFT's. Correlation functions in the CFT are computed by varying magnetic bulk sources, whereas correlation functions in the dual CFT are computed by electric bulk sources. Under massive deformations, the boundary effective action is generically minimized by massive self-dual configurations of the U(1) gauge field. We show that a self-dual choice of boundary conditions exists, and it corresponds to the self-dual topologically massive gauge theory in 2+1 dimensions. Thus, self-duality in three dimensions can be understood as a consequence of electric-magnetic invariance in the bulk of AdS4.