Jackson (From metaphysics to ethics, Oxford, Oxford University Press, 1998) argues that conceptual analysis plays a modest, albeit crucial, role in ‘serious metaphysics’: roughly, the project of demystifying phenomena we take to be mysterious by locating them in the natural world. This defence of conceptual analysis is associated with ‘the Canberra Plan’, a philosophical methodology that has its roots in the works of both Lewis (J Philos 67(13):427–446, 1970, Australas J Philos 50:249–258, 1972) and Jackson (Monist 77:93–110, 1994, 1998). There is, however, a distinction to be drawn between conceptual analysis, as it is typically employed in the Canberra plan, and a version of it defended by Jackson himself. In this paper, I elucidate this distinction, and employ examples from the history of science to argue the use of the former, but not the latter, incurs certain problems of conceptual change. Moreover, I also argue neither can be used to undertake serious metaphysics—the former because of the aforementioned problems, and the latter due to the machinery it employs to solve them.