Aggregating multiple probability intervals to improve calibration
Journal Title: Judgment and Decision Making - Year 2015, Vol 10, Issue 2
Abstract
We apply the principles of the “Wisdom of Crowds (WoC)” to improve the calibration of interval estimates. Previous research has documented the significant impact of the WoC on the accuracy of point estimates but only a few studies have examined its effectiveness in aggregating interval estimates. We demonstrate that collective probability intervals obtained by several heuristics can reduce the typical overconfidence of the individual estimates. We re-analyzed data from Glaser, Langer and Weber (2013) and from Soll and Klayman (2004) and applied four heuristics Averaging, Median, Enveloping, Probability averaging-suggested by Gaba, Tsetlin and Winkler (2014) and new heuristics, Averaging with trimming and Quartiles. We used the hit rate and the Mean Squared Error (MSE) to evaluate the quality of the methods. All methods reduced miscalibration to some degree, and Quartiles was the most beneficial securing accuracy and informativeness.
Authors and Affiliations
Saemi Park and David V. Budescu
Keywords
Related Articles
- EP ID EP678166
- DOI -
- Views 158
- Downloads 0
The effect of consumer ratings and attentional allocation on product valuations
Online marketplaces allow consumers to leave reviews about the products they purchase, which are visible to potential customers and competitors. While the impact of reviews on valuations of worth and purchasing decisions...
Actor/observer asymmetry in risky decision making
Are people willing to gamble more for themselves than what they deem reasonable for others? We addressed this question in a simplified computer gambling task in which subjects chose from a set of 10 cards. Subjects selec...
Risky Decision Making: Testing for Violations of Transitivity Predicted by an Editing Mechanism
Transitivity is the assumption that if a person prefers A to B and B to C, then that person should prefer A to C. This article explores a paradigm in which Birnbaum, Patton and Lott (1999) thought people might be systema...
Ideology, motivated reasoning, and cognitive reflection
Decision scientists have identified various plausible sources of ideological polarization over climate change, gun violence, national security, and like issues that turn on empirical evidence. This paper describes a stud...
Thoughtful days and valenced nights: How much will you think about the problem?
Considerable research has pointed towards processing differences as a viable means for understanding the strength and likelihood of a framing effect. In the current study we explored how differences in processing may eme...