[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v61y2013i6p1383-1398.html
   My bibliography  Save this article

The Wisdom of Competitive Crowds

Author

Listed:
  • Kenneth C. Lichtendahl

    (Darden School of Business, University of Virginia, Charlottesville, Virginia 22903)

  • Yael Grushka-Cockayne

    (Darden School of Business, University of Virginia, Charlottesville, Virginia 22903)

  • Phillip E. Pfeifer

    (Darden School of Business, University of Virginia, Charlottesville, Virginia 22903)

Abstract
When several individuals are asked to forecast an uncertain quantity, they often face implicit or explicit incentives to be the most accurate. Despite the desire to elicit honest forecasts, such competition induces forecasters to report strategically and nontruthfully. The question we address is whether the competitive crowd's forecast (the average of strategic forecasts) is more accurate than the truthful crowd's forecast (the average of truthful forecasts from the same forecasters). We analyze a forecasting competition in which a prize is awarded to the forecaster whose point forecast is closest to the actual outcome. Before reporting a forecast, we assume each forecaster receives two signals: one common and one private. These signals represent the forecasters' past shared and personal experiences relevant for forecasting the uncertain quantity of interest. In a set of equilibrium results, we characterize the nature of the strategic forecasts in this game. As the correlation among the forecasters' private signals increases, the forecasters switch from using a pure to a mixed strategy. In both cases, forecasters exaggerate their private information and thereby make the competitive crowd's forecast more accurate than the truthful crowd's forecast.

Suggested Citation

  • Kenneth C. Lichtendahl & Yael Grushka-Cockayne & Phillip E. Pfeifer, 2013. "The Wisdom of Competitive Crowds," Operations Research, INFORMS, vol. 61(6), pages 1383-1398, December.
    • Handle: RePEc:inm:oropre:v:61:y:2013:i:6:p:1383-1398
    DOI: 10.1287/opre.2013.1213
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.2013.1213
    Download Restriction: no
    File URL: https://libkey.io/10.1287/opre.2013.1213?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ottaviani, Marco & Sorensen, Peter Norman, 2006. "The strategy of professional forecasting," Journal of Financial Economics, Elsevier, vol. 81(2), pages 441-466, August.
    2. Kenneth C. Lichtendahl, Jr. & Robert L. Winkler, 2007. "Probability Elicitation, Scoring Rules, and Competition Among Forecasters," Management Science, INFORMS, vol. 53(11), pages 1745-1755, November.
    3. Oliver Kim & Steve C. Lim & Kenneth W. Shaw, 2001. "The Inefficiency of the Mean Analyst Forecast as a Summary Forecast of Earnings," Journal of Accounting Research, Wiley Blackwell, vol. 39(2), pages 329-335, September.
    4. Michael P. Keane & David E. Runkle, 1998. "Are Financial Analysts' Forecasts of Corporate Profits Rational?," Journal of Political Economy, University of Chicago Press, vol. 106(4), pages 768-805, August.
    5. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    6. R. Winkler & Javier Muñoz & José Cervera & José Bernardo & Gail Blattenberger & Joseph Kadane & Dennis Lindley & Allan Murphy & Robert Oliver & David Ríos-Insua, 1996. "Scoring rules and the evaluation of probabilities," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 5(1), pages 1-60, June.
    7. Osband, Kent, 1989. "Optimal Forecasting Incentives," Journal of Political Economy, University of Chicago Press, vol. 97(5), pages 1091-1112, October.
    8. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    9. Victor Richmond R. Jose & Robert L. Winkler, 2009. "Evaluating Quantile Assessments," Operations Research, INFORMS, vol. 57(5), pages 1287-1297, October.
    10. David Laster & Paul Bennett & In Sun Geoum, 1999. "Rational Bias in Macroeconomic Forecasts," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(1), pages 293-318.
    11. Clemon, Robert T & Winkler, Robert L, 1986. "Combining Economic Forecasts," Journal of Business & Economic Statistics, American Statistical Association, vol. 4(1), pages 39-46, January.
    12. Kay-Yut Chen & Leslie R. Fine & Bernardo A. Huberman, 2004. "Eliminating Public Knowledge Biases in Information-Aggregation Mechanisms," Management Science, INFORMS, vol. 50(7), pages 983-994, July.
    13. Robert T. Clemen & Robert L. Winkler, 1985. "Limits for the Precision and Value of Information from Dependent Sources," Operations Research, INFORMS, vol. 33(2), pages 427-442, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. David V. Budescu & Eva Chen, 2015. "Identifying Expertise to Extract the Wisdom of Crowds," Management Science, INFORMS, vol. 61(2), pages 267-280, February.
    2. Vincenz Frey & Arnout van de Rijt, 2021. "Social Influence Undermines the Wisdom of the Crowd in Sequential Decision Making," Management Science, INFORMS, vol. 67(7), pages 4273-4286, July.
    3. Sanjay Banerjee, 2021. "Does Competition Improve Analysts’ Forecast Informativeness?," Management Science, INFORMS, vol. 67(5), pages 3219-3238, May.
    4. Ho Cheung Brian Lee & Jan Stallaert & Ming Fan, 2020. "Anomalies in Probability Estimates for Event Forecasting on Prediction Markets," Production and Operations Management, Production and Operations Management Society, vol. 29(9), pages 2077-2095, September.
    5. Keppo, Jussi & Satopää, Ville A., 2024. "Bayesian herd detection for dynamic data," International Journal of Forecasting, Elsevier, vol. 40(1), pages 285-301.
    6. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    7. Fotios Petropoulos & Enno Siemsen, 2023. "Forecast Selection and Representativeness," Management Science, INFORMS, vol. 69(5), pages 2672-2690, May.
    8. Satopää, Ville A., 2021. "Improving the wisdom of crowds with analysis of variance of predictions of related outcomes," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1728-1747.
    9. Brown, Alasdair & Reade, J. James, 2019. "The wisdom of amateur crowds: Evidence from an online community of sports tipsters," European Journal of Operational Research, Elsevier, vol. 272(3), pages 1073-1081.
    10. Sepideh Bazazi & Jorina von Zimmermann & Bahador Bahrami & Daniel Richardson, 2019. "Self-serving incentives impair collective decisions by increasing conformity," PLOS ONE, Public Library of Science, vol. 14(11), pages 1-12, November.
    11. Cem Peker, 2023. "Extracting the collective wisdom in probabilistic judgments," Theory and Decision, Springer, vol. 94(3), pages 467-501, April.
    12. Jens Witkowski & Rupert Freeman & Jennifer Wortman Vaughan & David M. Pennock & Andreas Krause, 2023. "Incentive-Compatible Forecasting Competitions," Management Science, INFORMS, vol. 69(3), pages 1354-1374, March.
    13. Taylor, James W., 2021. "Evaluating quantile-bounded and expectile-bounded interval forecasts," International Journal of Forecasting, Elsevier, vol. 37(2), pages 800-811.
    14. Asa B. Palley & Jack B. Soll, 2019. "Extracting the Wisdom of Crowds When Information Is Shared," Management Science, INFORMS, vol. 67(5), pages 2291-2309, May.
    15. Yanwei Jia & Jussi Keppo & Ville Satopää, 2023. "Herding in Probabilistic Forecasts," Management Science, INFORMS, vol. 69(5), pages 2713-2732, May.
    16. Stefan Palan & Jürgen Huber & Larissa Senninger, 2020. "Aggregation mechanisms for crowd predictions," Experimental Economics, Springer;Economic Science Association, vol. 23(3), pages 788-814, September.
    17. Taylor, James W., 2020. "A strategic predictive distribution for tests of probabilistic calibration," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1380-1388.
    18. repec:grz:wpsses:2019-01 is not listed on IDEAS

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cem Peker, 2023. "Extracting the collective wisdom in probabilistic judgments," Theory and Decision, Springer, vol. 94(3), pages 467-501, April.
    2. Robert L. Winkler & Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose, 2019. "Probability Forecasts and Their Combination: A Research Perspective," Decision Analysis, INFORMS, vol. 16(4), pages 239-260, December.
    3. Asa B. Palley & Jack B. Soll, 2019. "Extracting the Wisdom of Crowds When Information Is Shared," Management Science, INFORMS, vol. 67(5), pages 2291-2309, May.
    4. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    5. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    6. Phillip E. Pfeifer, 2016. "The promise of pick-the-winners contests for producing crowd probability forecasts," Theory and Decision, Springer, vol. 81(2), pages 255-278, August.
    7. Robert L. Winkler & Robert T. Clemen, 2004. "Multiple Experts vs. Multiple Methods: Combining Correlation Assessments," Decision Analysis, INFORMS, vol. 1(3), pages 167-176, September.
    8. Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose & Robert L. Winkler, 2017. "Quantile Evaluation, Sensitivity to Bracketing, and Sharing Business Payoffs," Operations Research, INFORMS, vol. 65(3), pages 712-728, June.
    9. Marinovic, Iván & Ottaviani, Marco & Sorensen, Peter, 2013. "Forecasters’ Objectives and Strategies," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 690-720, Elsevier.
    10. Gneiting, Tilmann, 2011. "Quantiles as optimal point forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 197-207, April.
    11. Satopää, Ville A., 2021. "Improving the wisdom of crowds with analysis of variance of predictions of related outcomes," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1728-1747.
    12. Phillip E. Pfeifer & Yael Grushka-Cockayne & Kenneth C. Lichtendahl, 2014. "The Promise of Prediction Contests," The American Statistician, Taylor & Francis Journals, vol. 68(4), pages 264-270, November.
    13. Lahiri, Kajal & Yang, Liu, 2013. "Forecasting Binary Outcomes," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1025-1106, Elsevier.
    14. Jaspersen, Johannes G., 2022. "Convex combinations in judgment aggregation," European Journal of Operational Research, Elsevier, vol. 299(2), pages 780-794.
    15. Patrick Afflerbach & Christopher Dun & Henner Gimpel & Dominik Parak & Johannes Seyfried, 2021. "A Simulation-Based Approach to Understanding the Wisdom of Crowds Phenomenon in Aggregating Expert Judgment," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(4), pages 329-348, August.
    16. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios & Chen, Zhi & Gaba, Anil & Tsetlin, Ilia & Winkler, Robert L., 2022. "The M5 uncertainty competition: Results, findings and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1365-1385.
    17. Laurent L. Pauwels & Andrey L. Vasnev, 2017. "Forecast combination for discrete choice models: predicting FOMC monetary policy decisions," Empirical Economics, Springer, vol. 52(1), pages 229-254, February.
    18. Ottaviani, Marco & Sorensen, Peter Norman, 2006. "The strategy of professional forecasting," Journal of Financial Economics, Elsevier, vol. 81(2), pages 441-466, August.
    19. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    20. Wojciech Olszewski & Alvaro Sandroni, 2008. "Manipulability of Future-Independent Tests," Econometrica, Econometric Society, vol. 76(6), pages 1437-1466, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:61:y:2013:i:6:p:1383-1398. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.