On the Challenges of Measurement in the Human Sciences
Measurement practices are central to most sciences. In the human sciences, however, it remains controversial whether the measurement of human attributes—depression, happiness, intelligence, etc.—has been successful. Are, say, widely used depression questionnaires valid measuring instruments? Can we trust self-reported happiness scales to deliver quantitative measurements as it is sometimes claimed? These and related questions are till today hotly disputed.
There are two main frameworks under which human measurements are studied and criticized. One is the so-called construct validity framework. Here, criticisms to human measurements are typically of the form “this instrument is not valid: it does not actually measure the attribute we set out to measure”. The second framework is the standard typology of measurement scales (nominal, ordinal, interval, and ratio). Human measurements are commonly challenged for being merely ordinal—not quantitative: interval or ratio—despite the fact that many researchers use measurement results as if they were quantitative.
The first part of the dissertation studies how adequate these frameworks are for evaluating measuring instruments and the inferences they afford. Regarding the concept of validity, it is commonly understood unconditionally, that is, without restricting validity judgments about measuring instruments to context-specific situations. Instruments are said to be (in)valid simpliciter. In Chapter 2, I argue against this conception of validity and in favor of a contextual one. Regarding the second main framework, the standard typology of scales is typically linked to a set of prescriptions regarding (un)justified measurement inferences. I call this classification-cum-prescriptions the “received view” on measurement scales. In chapters 3 and 4, I question the idea that the received view is an impeccable guide for clarifying the kinds of inferences we are licensed to make from measurement results in human science contexts. I motivate general doubts about the adequacy of the received view in Chapter 3, and I articulate these doubts in detail for the specific case of average group comparisons in Chapter 4. The upshot of this first part of the dissertation is a deeper awareness of the complexity surrounding which inferences can legitimately be made from measuring instruments.
The first part of the dissertation is largely framed under the assumption that some human attributes are indeed quantitative, even if they are not currently being measured quantitatively. The second part addresses two important issues raised by that assumption. Chapter 5 tackles the so-called “quantity objection”: that human science attributes are themselves not quantitative, thus they cannot be quantitatively measured. This objection has been deployed to argue against optimistic positions regarding human quantification. I argue that the quantity objection is not successful in this sense—it begs the question to these optimistic human scientists that, just like their colleagues in the physical sciences have done, postulate theoretical quantitative attributes as working hypotheses. But what does treating human attributes quantitatively as working hypotheses amount to? And what are, or can be, “amounts” of depression, happiness, etc.? Chapter 6 argues that there is not one but various approaches for quantitatively conceiving of human attributes, each with its own success conditions. This chapter offers a conceptual framework for making progress on debates about controversial human measurements.