Aggregating multiple probability intervals to improve calibration
Journal Title: Judgment and Decision Making - Year 2015, Vol 10, Issue 2
Abstract
We apply the principles of the “Wisdom of Crowds (WoC)” to improve the calibration of interval estimates. Previous research has documented the significant impact of the WoC on the accuracy of point estimates but only a few studies have examined its effectiveness in aggregating interval estimates. We demonstrate that collective probability intervals obtained by several heuristics can reduce the typical overconfidence of the individual estimates. We re-analyzed data from Glaser, Langer and Weber (2013) and from Soll and Klayman (2004) and applied four heuristics Averaging, Median, Enveloping, Probability averaging-suggested by Gaba, Tsetlin and Winkler (2014) and new heuristics, Averaging with trimming and Quartiles. We used the hit rate and the Mean Squared Error (MSE) to evaluate the quality of the methods. All methods reduced miscalibration to some degree, and Quartiles was the most beneficial securing accuracy and informativeness.
Authors and Affiliations
Saemi Park and David V. Budescu
Keywords
Related Articles
- EP ID EP678166
- DOI -
- Views 160
- Downloads 0
Welfare effects of nudges: The emotional tax of calorie menu labeling
Traditionally, information has been assumed to never harm consumers, a notion recently challenged. Salience nudges have been argued to evoke negative emotions, therefore acting as “emotional taxes”. I design a hypothetic...
Simple eye movement metrics can predict future decision making performance: The case of financial choices
Decisions are often delegated to experts chosen based on their past performance record which may be subject to noise. For instance, a person with little skill could still make a lucky decision that proves correct ex-post...
A new intuitionism: Meaning, memory, and development in Fuzzy-Trace Theory
Combining meaning, memory, and development, the perennially popular topic of intuition can be approached in a new way. Fuzzy-trace theory integrates these topics by distinguishing between meaning-based gist representatio...
Willingness to test for BRCA1/2 in high risk women: Influenced by risk perception and family experience, rather than by objective or subjective numeracy?
Genetic testing for breast and ovarian cancer can help target prevention programs, and possibly reduce morbidity and mortality. A positive result of BRCA1/2 is a substantial risk factor for breast and ovarian cancer, and...
Cultivating credibility with probability words and numbers
Recent research suggests that communicating probabilities numerically rather than verbally benefits forecasters’ credibility. In two experiments, we tested the reproducibility of this communication-format effect. The eff...