James Joyce (Michigan): “Accuracy, Updating, and the Choice of Scoring Rules”

Abstract: Proponents of an accuracy-centered epistemology have argued that proper scoring rules can be used to assess the accuracy of degrees of belief (or “credences”), and have suggested that certain core epistemic norms for credecnces can be understood and justified with the help of such rules. Many who go this route are attracted to the idea that revising credences in light of new evidence should proceed by a process that involves “mechanical” updating. The idea is that, upon receiving new data, an agent should always move to the credal state, among those consistent with that data, which maximizes expected accuracy. This provides a rationale for Bayesian conditioning in contexts where the new data involves learning some proposition with certainty. However, for less conclusive experiences it can be shown that each proper score has its own characteristic update rule. For example, the so-called logarithmic score has Jeffrey conditioning as its characteristic update, while H. Leitgeb and R. Pettigrew have shown that the Brier (quadratic loss) score has a completely different characteristic update. Since there are good epistemic reasons to prefer Jeffrey conditioning to any other update rule (in the contexts where it applies), it looks as we must either jettison mechanical updating or embrace the log score as the one true measure of epistemic accuracy. Neither option is appealing: the accuracy-centered approach seems deeply committed to mechanical updating, but the approach’s appeal diminishes quite drastically if it is forced to single out some particular score as correct. Moreover, as I will argue, the log score has limitations when it comes to updating; it classifies far too many things as learning experiences (in the sense of Skyrms), and fails to recognize that certain sorts of belief changes, even reliable ones, can decrease overall accuracy. Fortunately, there is no real dilemma here. As I will show, a proper application of the accuracy-centered updating will mandate Jeffrey conditioning as the uniquely correct belief revision rule in all contexts where it can be applied. While making the case for this conclusion I will discuss some recent work by Peter Lewis and Dan Fallis, which seems to suggest that no proper scoring rule can be used as a basis for updating.

James Joyce is a professor of philosophy and of statistics at the University of Michigan in Ann Arbor. His research interests include rational choice theory, causal reasoning, Bayesian approaches to statistics and inductive inference, and the use of “imprecise” probabilities to model belief states.

#LSEChoiceGroup