Evaluation of theories vs. validation of hypotheses

Research should be presented with appropriate choice of words to the world. So it bugs me if researchers, maybe unknowingly, overreach and call the evaluation of a theory a validation thereof. I don’t think you can ever fully validate a theory, you can only validate individual hypotheses.

The following figure shows how I think key terms should be used.

The two main dimensions that determine choice of words are open vs. closed (world) and incremental vs. final conclusion:

1. Open vs. closed. The most fundamental distinction is whether your claims are for an open or a closed world. Only mathematics really provides a closed world. There you can talk about a proof. In an open world, you cannot talk about having proved anything correct.
2. Incremental vs. conclusive. In an open world, any contribution to our knowledge about the validity of a theory is incremental; theories only stand until they are proved wrong [1]. Some theories stand a long time (e.g. Newtonian laws) but eventually even they are replaced.

In a closed world, you can prove a theorem. In an open world, you can validate a hypothesis, and you can evaluate a theory. You cannot prove an (open world) theory nor a hypothesis, and you also can’t validate a theory.

The reason is the relationship between a theory and the hypotheses generated from it. For a given hypothesis, we have well established research methods like hypothesis-testing surveys and controlled experiments that let us show a hypothesis holds or not, i.e. is validated or invalidated within reasonable limits of certainty as defined by the employed research method.

For a theory, all we have is an evaluation thereof by continuously testing the hypotheses generated from it. There is no single comprehensive research methodology that affords us the same certainty about validity as we currently have for single hypotheses. Hence, we can only incrementally evaluate a theory by piling up conclusively validated hypotheses generated from the theory.

---

[1] The irony of arguing you can’t proof a theory right but you can proof it wrong does not escape me. Theory of science as of today argues you can’t even proof a theory wrong. However, in the rather traditional mindset of computer science this is still the appropriate wording.