Sense and nonsense in psychological measurement: A case of problem and method passing one another by

Abstract

There exists conceptual confusion in the psychological measurement literature. One such example is the nonsense of the question, “are psychological attributes quantitative?” when referring to nontechnical psychological phenomena. Rather than debating whether this is an empirical or conceptual question, the issue of concern should be whether it makes sense to ask it. As illuminated by Wittgenstein almost a century ago, the concepts of psychology are everyday concepts rather than technical ones. However, the enduring comparison of psychology with the physical sciences leads to the adoption of technical language that loses its meaning when used with the aim of explaining everyday concepts. For this reason, asking whether psychological attributes are quantitative is, in many cases, a nonsensical question to ask.

The confusion and barrenness of psychology is not to be explained by its being a “young science”; its state is not comparable with that of physics, for instance, in its beginnings. . . For in psychology, there are experimental methods and conceptual confusion. . .

The existence of the experimental method makes us think that we have the means of getting rid of the problems which trouble us; but problem and method pass one another by. (Wittgenstein, 1953, p. 243e)

In considering the problems of psychological measurement, Franz (2022) demonstrates that Wittgenstein’s (1953) observation of “problem and method pass[ing] one another by” (p. 243e) remains as valid today as it was in the previous century. In my view, the primary takeaway from Franz’s (2022) critique is that the question, “are psychological attributes quantitative?” is nonsensical when referring to nontechnical psychological phenomena. While I agree with many points made by Franz (2022), I argue that the conceptual issue at stake is not whether this question is an empirical or conceptual one, but rather, whether it makes sense to ask it. I examine Wittgenstein’s remarks regarding conceptual confusion in psychology, highlighting how the tendency of psychologists to slip between scientific and everyday uses of language leads to nonsense. Although the relevance of these topics for psychological measurement was detailed more than two decades ago by Maraun (1998), the ongoing conceptual confusion in the psychological measurement literature warrants further examination.

The roots of the conceptual problem: Psychology as a “young science”

To fully appreciate the relevance of Wittgenstein’s (1953) observation that, “in psychology, there are experimental methods and conceptual confusion” (p. 243e), it is important to consider the context within which he was making this observation. Hacker (1996) located the origin of Wittgenstein’s remarks as being a response to ideas presented by Köhler in his book, Gestalt Psychology (1929/1947). Köhler (1929/1947) discussed the evolution of physics as a natural science, giving special attention to “the transformation of qualitative observation into quantitative measurement by means of sophisticated techniques and instruments” (Hacker, 1996, p. 111). For example, he described the indirect quantitative measurement of “atmospheres,” “volts,” and “temperatures” through “the location of a visual line (a pointer) on a scale of other visual lines” (Köhler, 1929/1947, p. 35). Köhler (1929/1947) compared these advances in physics with the burgeoning works of Behaviourists, noting that their efforts to attain objective observations indicated improvement from the previously subjective dominance of introspective methods in the field. However, he also cautioned against the Behaviourists’ dismissal of direct experience. Psychology was still considered a young science at the time, and, for Köhler (1929/1947), it therefore must continue to imitate other, more advanced, sciences in their infancy. This required establishing correlations between qualitative descriptions and underlying cognitive processes.

For Wittgenstein (1953), psychology was not comparable with the physical sciences in their infancy. This is because “the concepts of psychology are just everyday concepts. They are not concepts newly fashioned by science for its own purpose, as are the concepts of physics and chemistry” (Wittgenstein, 1980, p. 12e). Hacker (1996) elaborated on this point by questioning whether psychological concepts such as “believing,” “fearing,” or “intending” could one day be replaced by technical concepts; however, he contended that, “with their elimination, one would arguably eliminate the very explananda of the science of psychology” (Hacker, 1996, p. 113). For example, if one were to replace everyday senses of “belief” with a technical sense of the term, they would be dealing with either a very restricted or an entirely new concept. This may be harmless if the aim of psychology is to understand such concepts in a restricted technical sense; however, it would lead to conceptual confusion if the aim were to understand belief in its everyday sense.

“Seeing, hearing, thinking, feeling, willing, are not the subject of psychology in the same sense as that in which the movements of bodies, the phenomena of electricity etc., are the subject of physics” (Wittgenstein, 1953, p. 159e). The emphasized part of this passage is of crucial significance. Wittgenstein (1953, 1980) often talked of sense and nonsense in language use, demonstrating that psychologists frequently use statements consisting of literal nonsense. This is evident when psychologists adopt language from the physical sciences, including quantitative description. For example, it is nonsense to speak of adding my belief with your belief to attain a total amount of belief. The root of this conceptual confusion lies in the comparison of psychology with the physical sciences, as this leads to the adoption of technical language that loses its meaning when used with the aim of explaining everyday concepts.

Senses of measurement

To examine how language use in the psychological measurement literature leads to conceptual confusion, one must consider the various ways in which the concept “measurement” is used. Michell (2004) provided an extensive historical review, demonstrating that, in psychology, the dominant use of the measurement concept stems from an operationist view in which the meaning of a concept is taken to be inherent to the specific operations that determine it. However, the question, “are psychological attributes quantitative?” follows from the classical conception that “all measurable attributes are quantitative,” which commits to a realist, rather than operationist view (Michell, 2004, p. 25, 2005). As such, there appear to be at least two differing senses of measurement that are of relevance to the current discussion.

Although Franz (2022) is critical of Michell’s emphasis on measurement as an empirical, rather than conceptual, phenomenon, it is important to note that Michell has also previously been critical of conceptual confusion in psychology. For example, Michell (2004) argued that if scientists within a given domain misuse the measurement concept, “then the methods called ‘measurement’ within that science would not disclose the sorts of facts about the world that they might be thought to and those scientists would misunderstand what they were doing” (p. 3). Michell’s (2004) concern appears to be with psychometricians adopting an incorrect definition of measurement. From a realist perspective, “real numbers are ratios of magnitudes of a quantity” (Michell, 2004, p. 62). To engage in measurement is to “discover facts about magnitudes” (Michell, 2004, p. 119). Prior to doing so, one must answer the question of whether an object has quantitative attributes to discover. This is conceptualized as an empirical question because quantity, as a “system of magnitudes,” is taken to be discoverable (Michell, 2004, p. 119). Under this view, misusing the measurement concept leads to confusion because the application of measurement becomes misleading.

The dominant use of the measurement concept in psychology, which Michell (2004) has critiqued, stems from Stevens’ (1946) definition of measurement as “the assignment of numerals to objects or events according to rules” (p. 677). Michell (2004) has shown that Stevens was inspired by both operationist and representationalist philosophy in promoting this definition. In attempting to show that all mathematics could be reduced to symbolic logic, philosophers upholding a representationalist view (e.g., Russell, 1903) conceptualized number and quantity as being separate from one another. Thus, numbers are used as representations of empirical relations. Stevens (1946) extended this tenet of representationalism to psychological measurement, positing that all cases in which numerical representations are made according to rules are cases of measurement. From this perspective, measurement amounts to numerical representation, as numbers are taken as symbols that denote empirical facts.

According to Michell (2004), Stevens’ (1946) definition leads to conceptual confusion because it allows psychologists to bypass the empirical question of whether a phenomenon has quantitative attributes. Wittgenstein also famously rejected the representationalist view, but for different reasons. For Wittgenstein (1953), language does not provide a picture of facts, and meaning is not inherent in the object being described. Rather, meaning arises from the use of language within a time and place. The problem is the lack of care in paying attention to language use. While Michell and Franz (following Wittgenstein and Hacker) appear to agree that psychometricians are conceptually confused, they differ with respect to the reason why, as well as to the solution to this problem. For Michell, it appears that a solution would require psychologists to empirically answer the question “are psychological attributes quantitative?” In contrast, a Wittgensteinian perspective entails questioning whether it is sensible to adapt the technical language of the sciences for use with everyday psychological concepts. In other words, does it make sense to ask whether psychological attributes are quantitative?

Does it make sense to ask whether psychological attributes are quantitative?

“Language-game” is a concept introduced by Wittgenstein to demonstrate the ways in which “the speaking of language is part of an activity, or of a form of life” (Wittgenstein, 1953, p. 15e). Drawing an analogy between language and games illustrates that how we use language is important for establishing the meanings of concepts. In the same way that one can establish the meaning of pieces in a game through description of its rules, one can also establish the meaning of concepts in a language through description of its grammar (Baker & Hacker, 2009). For example,

I am explaining chess to someone; and I begin by pointing to a chess piece and saying “This is the king; it can move in this-and-this way”, and so on. – In this case we shall say: the words “This is the king”. . . are an explanation of a word only if the learner already ‘knows what a piece in a game is’. That is, if, for example, he has already played other games, or has watched ‘with understanding’ how other people play – and similar things. (Wittgenstein, 1953, p. 19e)

Establishing the meaning of the king in a game of chess is not done by correlating the term king to the object it represents or through an empirical discovery of the meaning of “king,” rather, it is a part of the activity of learning and playing the game. Doing so teaches us what the king means in the context of the game and the possible sensible (and nonsensical) uses of the term.

Like the king in Wittgenstein’s chess example, we might consider the possible uses of the measurement concept within the language-games of science. This consideration is trickier than consideration of the king in chess, as the language-games of science are far more complex and nuanced. There exist multiple possible uses of the measurement concept in science discourse (including some that are incompatible), and thus, there are also multiple possible meanings of the term. For instance, when a physical scientist adopts the realist view of measurement, describing it as a process by which one captures magnitudes of quantities, they imply a different meaning of measurement than an empirical psychologist who quantifies happiness through numerical representation and describes this process as measurement.

Michell (2004, 2005) rightfully notes that espousing a theory of science consistent with the realist view while also adopting a definition of measurement inconsistent with the realist view is conceptually problematic. However, the confusion is not due to the use of an incorrect definition of measurement, per se. Rather, the confusion arises when one considers the context of use—it being one in which psychologists uphold a standard of science in which empirical discovery of real-valued quantities is the aim. It is as if one intended to play the game of chess in its classical form but introduced new rules for the movement of the king. “If you change the rules, you change the game” (Baker & Hacker, 2009, p. 47). Psychologists, in adopting their own use of the measurement concept, while aiming to play the same measurement “game” as physical scientists, land in conceptual confusion. However, as illustrated by Franz (2022) and Maraun (1998), the confusion goes beyond this transgression when psychologists also attempt to incorporate everyday psychological concepts into this endeavour, while also maintaining their everyday uses.

It may make sense, in the context of the realist view, to interpret the question of whether the attributes of physical phenomena are quantitative as an empirical question. Indeed, as Michell (2004, 2005) has argued, it is sensible to apply Hölder’s axioms to the properties of physical phenomena such as length or mass. What Franz (2022) illustrates in his conceptual analysis is that this question often becomes nonsensical when applied to psychological attributes, as psychologists who engage in measurement practices wish to make inferences that reside in the realm of everyday language. For example, developmental psychologists studying “expectation” and “surprise” in infants wish to employ these concepts in their everyday senses (as in, “the infant expected the puppet to help, but she was surprised to see that it did not”). They must therefore be cautious of slipping back and forth between a technical sense of “expectation,” operationally defined as the infant’s looking time within a controlled laboratory setting, and an everyday sense of the term in which a reduction to looking time would be nonsensical (Tafreshi et al., 2014). Moreover, Franz (2022) demonstrates that, as is the case with other psychological concepts, asking whether “expectation” has quantitative attributes would be literal nonsense.

If Köhler (1929/1947) was correct that psychology is a young science akin to physics in its beginnings, then the question of whether psychological attributes are quantitative might be a sensible one to ask. However, unlike the physical sciences, psychology is a discipline concerned primarily with studying familiar everyday concepts. By questioning whether speaking of quantitative psychological attributes is sensible, Franz (2022) reminds psychologists to “pay attention to your nonsense” (Wittgenstein, 1998, p. 64e).