We study linear inverse problems under the premise that the forward operator is not at hand but given indirectly through some input-output training pairs. We demonstrate that regularization by projection and variational regularization can be formulated by using the training data only and without making use of the forward operator. We study convergence and stability of the regularized solutions in view of T. I. Seidman. "Nonconvergence Results for the Application of Least-Squares Estimation to Ill-Posed Problems". Journal of Optimization Theory and Applications 30.4 (1980), pp. 535-547, who showed that regularization by projection is not convergent in general, by giving some insight on the generality of Seidman's nonconvergence example. Moreover, we show, analytically and numerically, that regularization by projection is indeed capable of learning linear operators, such as the Radon transform.