We may not know what we are doing…

From my excursion into qualitative research land (the aforementioned Berliner Methodentreffen) I took away some rather confusing impressions about the variety of what people consider science. I’m well aware of different philosophies of science (from positivism to radical constructivism) and their impact on research methodology (from controlled experiments to action research, ethnographies, etc.) I did not expect, however, for people to be so divided about fundamental assumptions about what constitutes good science.

One of the initial surprises for me was to learn that it is acceptable for a dissertation to apply only one method and for that method to only deliver descriptive results (and thereby not really make a contribution to theory). In computer science, it is difficult to publish solely theory development research (let alone purely descriptive results) without any theory validation attempt, even if only selective. The limits of what can be done in 3-5 Ph.D. student years are clear, but this shouldn’t lead anyone to lower expectations.

A related surprise came when people talked about their work and how they applied two different methods, with two different results. In the cases I listened to, the authors happily presented these two separate and conflicting results without any attempt of consolidating them into one description or theory. Again, in computer science, if I were to claim “I found a problem” I would be expected to at least attempt to find a solution as well. There is no doubt that such consolidation might be difficult, but again, you can’t really throw in the towel either.

Not surprisingly, talking about theory validation riled some participants. An immediate assumption was that by doing so I would assume one could uncover eternal unbridled truths. I don’t see how that follows. Ever since Popper, researchers have been well aware that their theory is only as good as it doesn’t get invalidated (falsified), often by applying the very methods that put the theory onto the playing field in the first place. To question theory validation is to argue that intersubjective communication and agreement is not possible and that an external people-independent reality does not exist. That may be true when talking to a schizophrenic person, but I don’t see how it applies to running a factory.

Sadly, the closing keynote confirmed a bias against theory validation. Paraphrasing from memory, the speaker declared to much cheer: “We may not know what we are doing, but we surely now our enemy.” With this he apparently meant anyone caring about inter-subjectively shared theories and their (always imperfect) validation.