Choosing how to represent knowledge effectively is a long-standing open problem. Cognitive science has shed light on the taxonomisation of representational systems from the perspective of cognitive processes, but a similar analysis is absent from the perspective of problem solving, where the representations are employed. In this paper we review how representation choices are made for solving problems in the context of theorem proving from three perspectives: cognition, heterogeneity, and computational demands. We contrast the different factors that are most important for each perspective in the context of problem solving to produce a list of considerations for developers of problem solving tools regarding representations that are appropriate for particular users and effective for specific problem domains.