Higher-Order Evidence in Science

Some Problematic Consequences of Steadfastness and Level-Splitting

Abstract

Despite our best efforts, we often fail to act in a perfectly rational manner. Recently, some epistemologists have suggested that we should admit our failings and develop a modest epistemology that would take our fallibility seriously. This includes accounting for the role of evidence of our irrationality, usually called higher-order evidence. It seems intuitive that modest reasoners should take such evidence into account. However, it turns out that incorporating higher-order evidence into a principled theory of what rationality requires is not an easy task. In this paper, I first review the debate about higher-order evidence, describing in detail the puzzle of higher-order evidence and the main positions about it in the literature. Then, I provide two novel examples of higher-order evidence, taken from science. I argue that these examples put pressure on the views that reject the role of higher-order evidence. These views commit themselves to the conclusion that some common scientific practices, such as evaluating evidence in systematic reviews or even running null hypothesis significance tests, are irrational.