I have been reading an article as cited bellow by M.A. Westereman (2006). I wrote this post as a series of responses to and summaries of points in that article regarding questions of objectivity in quantitative inquiry and how hermeneutics can provide an alternative to operationalization. Of particular interest are these topics in view of the new field of Learning Analytics. I come to the conclusion that concreteness as property of meaning (as opposed to meaning as abstraction) is also an important point to define in light of the limited opportunities for direct observation in online learning inquiry.
Quantitative research in social sciences and educational inquiry is irreducibly interpretive because the observer is not an objective spectator, but rather a participant in meaningful and meaning-laden activities in the world.
“psychological phenomena is irreducibly interpretive, it does not point to rejecting systematic quantitative research methods” (Westerman, 2006, p. 190).
“The key point depends on why one believes that interpretive inquiry is a good thing to begin with. My approach to this question is based on a hermeneutic philosophical perspective that takes practical activity as its starting point. This perspective draws on (Merleau-Ponty, 1962), (Wittgenstein, 1968), (Heidegger, 1962), and also on the pragmatists, especially (Dewey, 1896). It rejects the idea of a fundamental split between subject and world, which has been central to our philosophical tradition at least since Descartes. As (Toulmin, 1982) pointed out, according to this idea, the person is a ‘‘spectator.’’ Instead of beginning with the idea that person and world are the two poles of a basic dichotomy, the perspective that guides my remarks takes as its starting point what Merleau-Ponty (1962) called involved subjectivity. The person is always already involved in meaningful practical activities in the world, not a spectator fundamentally separate from the world” (Westerman, 2006, p. 196).
Operationalizing variables does not objectively transform theory into methodology.
“Notwithstanding nearly ubiquitous references to ‘‘operationalizing’’ variables and hypotheses about relations between variables, quantitative research procedures as they are actually employed do not objectively translate theoretical ideas about constructs and processes into meaning-free language about procedures” (Westerman, 2006, p. 191).
Quantitative research methods are not inherently more objective than any other type of methodologies that are accepted in peer-reviewed social science.
“Unfortunately, when psychologists subscribe to the official picture of quantitative research, they are adopting a misunderstanding of themselves as investigators on the basis of an outmoded philosophical viewpoint” (Westerman, 2006, p. 191).
Measurement of so-called latent variables often involves reducing rich meanings of human behavior down to abstractions defined by Likert-scales.
“Interpretation plays a role when it comes to measurement, which lies at the heart of quantitative research. This point is obvious regarding research based on clinical judgments about such global constructs as ‘‘irritable’’ or ‘‘submissive.’’ Research of this sort may employ the technical machinery of Likert-type scales or Q-sort procedures, but it clearly is based on rich appreciation of the meanings of human behavior” (Westerman, 2006, p. 191).
Operational definitions are abstractions that require interpretation and are often “natural language definitions.”
“In fact, what instruments of this kind provide by way of so-called ‘‘operational definitions’’ are natural language explanations of each category and examples. Such definitions are very useful, but they are anything but exhaustive. Indeed, they are useless if not employed by a coder with a wealth of background knowledge about the concepts, interpersonal behavior in our culture, and family life as we are familiar with it”(Westerman, 2006, p. 192).
Subjectivity: deciding what constructs to measure is not pure induction, but rather a result of interest; we choose ones that inform the theories we feel are at the center of our line of inquiry.
“Interpretation also enters into measurement when investigators employ such so-called ‘‘objective’’ measures as instruments for assessing intelligence and physiological recordings. Considerations about meaning play a central (although again, typically unacknowledged) role in determining what constructs we want to measure. We do not choose to assess abilities at ‘‘abstract reasoning’’ or ‘‘anxiety’’ as the result of a process of pure induction. Rather, we are interested in these concepts because of the role they play in our theories and, more importantly, because they have significance to us given what Merleau Ponty (1962) called our prereflective understanding, that is, a pre-theoretical appreciation of the day-to-day activities that make up our lives (more on this later)” (Westerman, 2006, p. 192).
There are no objective benchmarks, just implicit meanings that a researcher either accepts or rejects in the interpretive enterprise.
“Again, much more than induction is involved and there are no ‘‘final benchmarks.’’ Instead, researchers’ efforts are guided by a usually implicit appreciation of meanings as they engage in the complex, interpretive, and anything but transparent process of examining such matters as whether scores on the measure correlate with certain other measures” (Westerman, 2006, p. 193).
Hermeneutics as an alternative to operationalism: observation of meaningful practice in terms that are meaningful to the object of study, not in terms defined by a convention outside the object of study.
“Our accounts must always refer to what people are doing, that is, to meaningful practices, rather than attempt to fully explain the meaning involved in what they are doing in some other terms” (Westerman, 2006, p. 197).
The concreteness of meaning:
“Practical activity is bedrock. As Wittgenstein (1958, p. 226) put it, ‘‘forms of life’’ are ‘‘the given’’” (Westerman, 2006, p. 197).
The philosophical tradition of psychology gives many practitioners the idea that they can imposes abstractions (metaphors such as: information, memory, repositories, retrieval, storage etc.) upon the observational data that they collect and follow up by interpreting the data as evidence that these abstractions exist.
“According to the tradition—rationalism, in particular—meaning refers to abstract structures that lie behind the diversity of events. Philosophers proceeding along the lines of the tradition locate the capacity to appreciate such meanings in the subject’s mind, and psychologists follow suit with ideas about how there are such abstract structures as scripts or rules inside the mind” (Westerman, 2006, p. 198).
Empiricists in psychology reduce the meaning of phenomena and behavior to frequency counts and other types of observational data as if the meaning stopped at its operational definition.
“The empiricist wing of the tradition offers an alternative view, the idea that the apparent meaningfulness of events can be reduced to chains of brute occurrences, behaviors, and sense data. Psychologists turn to this idea when they argue that they can operationalize constructs and hypotheses” (Westerman, 2006, p. 198).
The problem with many assumptions in psychology comes down to the idea that an object’s nature (with or without meaning) can be determined by a removed observer.
“Note that these two positions about meaning share something in common. They represent different ways of maintaining that the nature of objects is such (whether that nature is characterized by abstract meanings or the absence of meaning) that a subject reflecting on those objects from a removed vantage point could arrive at what Wittgenstein (1958) called crystalline understanding, that is, a complete, determinate account” (Westerman, 2006, p. 198).
Meanings are concrete, not abstract, because they are located in the act of participating in the practical activity of everyday life. No one can step outside themselves, so meaning is always “inside” the life situation of time, place, and yourself.
“Meanings are not appreciated by a subject who reflects on things from the ‘‘outside’’ and recognizes abstract structures behind everyday events. People in general and psychologists in particular are participants in the world of practical activities from the outset. We appreciate the meaning of objects of inquiry from the ‘‘inside.’’ Their significance always refers to the roles they play in the world of practices in which we are already engaged. This locates meaning in the world (not the dead world of brute events, but the living world of practices), not in the mind. As a result, meanings are concrete, not abstract—an impossibility from the point of view of the philosophical tradition, but a central tenet of a perspective that takes practical activity as the starting point” (Westerman, 2006, p. 198).
For Learning Analytics, the implications of meaning as concreteness and the notion of “practical activity as the starting point” (Westerman, 2006, p. 198) may be complicated. For instance, meaning is concrete in the everydayness of 1. time (all experience is lived), 2. place (it happens somewhere with affordances available to the actors and within the nuances that the actors may or may not be aware of), 3. the agent (the person or persons doing the doing), and 4. the activity (what the agents are doing in this moment of everyday life). Under my definition of concreteness, environments that lend themselves to direct observation (e.g. classrooms, shop floors, etc.) seem a natural fit for inquiry that is intended to illuminate meaning. Design research (S. Barab & K. Squire, 2004; Sasha A. Barab, Thomas, Dodge, Kurt Squire, & Newell, 2004; Brown, 1992) runs in a similar vein.
On the other hand, in online or blended learning environments (i.e. a higher education class) time and place of activity are not easily defined or directly observable for the group of learners. The everydayness of the learners’ educational experiences is often intermittent and is always mediated through technology. The lived experience of the class may be synchronous at times and asynchronous at others. Questions arise such as, when the class is the unit of analysis, can everydayness be “observed” outside synchronous class sessions? How are the assumptions and constructs different for the abstractions used to interpret historical observational data vs. the assumptions and constructs behind how one observes a live session? How can the same thing be measured in the same way in both of those environments? If they can’t, how is meaning derived from concreteness across these two different constructs? Is observing a live, online collaboration technology like Breeze concreteness (It is lived experience but no one is really there, the are all just observing and participating remotely)? In this situation would you also have to observe them observing and participating remotely? If working with log data and blog posting is not concreteness, is it abstraction either?
The place is composed of a set of remote networks that connect class members’ communication technologies together. These people’s access to the internet not only serves as the educational venue but also serves as the same entry point for other activities in these people’s lives, such as work, socializing, entertainment, etc. In a normal classroom environment, students are immersed in the physical and social conventions of school. For online higher ed. students, school is just one touch point in a larger array of life activities at any point in an educational exercise or activity.
Communities of practice, Socio-cultural Activity Theory and other areas of inquiry that adopt some of the hermeneutical assumptions will have to address the nature of meaning in inquiry within online learning contexts. At this point, it does not seem entirely clear how they maintain their fidelity of concreteness in any other situations besides those that are directly observable.
Barab, S., & Squire, K. (2004). Design-Based Research: Putting a Stake in the Ground. THE JOURNAL OF THE LEARNING SCIENCES, 13(1), 1–14.
Barab, S. A., Thomas, M. K., Dodge, T., Squire, K., & Newell, M. (2004). Critical Design Ethnography: Designing for Change. Anthropology & Education Quarterly, 35(2), 254-268. doi:10.1525/aeq.2004.35.2.254
Brown, A. L. (1992). Design Experiments: Theoretical and Methodological Challenges in Creating Complex Interventions in Classroom Settings. THE JOURNAL OF THE LEARNING SCIENCES, 2(2), 141–178.
Dewey, J. (1896). The reflex arc concept in psychology. Psychological Review, 3, 357-370.
Heidegger, M. (1962). Being and Time (Reprint.). Harper.
Merleau-Ponty, M. (1962). Phenomenology of Perception (n.e.). Routledge.
Toulmin, S. E. (1982). The return to cosmology: postmodern science and the theology of nature. University of California Press.
Westerman, M. A. (2006). Quantitative research as an interpretive enterprise: The mostly unacknowledged role of interpretation in research efforts and suggestions for explicitly interpretive quantitative investigations. New Ideas in Psychology, 24(3), 189-211. doi:10.1016/j.newideapsych.2006.09.004
Wittgenstein, L. (1968). Philosophical investigations. Macmillan.