[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2312.09081.html
   My bibliography  Save this paper

Forecasting skill of a crowd-prediction platform: A comparison of exchange rate forecasts

Author

Listed:
  • Niklas Valentin Lehmann
Abstract
Open online crowd-prediction platforms are increasingly used to forecast trends and complex events. Despite the large body of research on crowd-prediction and forecasting tournaments, online crowd-prediction platforms have never been directly compared to other forecasting methods. In this analysis, exchange rate crowd-predictions made on Metaculus are compared to predictions made by the random-walk, a statistical model considered extremely hard-to-beat. The random-walk provides less erroneous forecasts, but the crowd-prediction does very well. By using the random-walk as a benchmark, this analysis provides a rare glimpse into the forecasting skill displayed on open online crowd-prediction platforms.

Suggested Citation

  • Niklas Valentin Lehmann, 2023. "Forecasting skill of a crowd-prediction platform: A comparison of exchange rate forecasts," Papers 2312.09081, arXiv.org.
    • Handle: RePEc:arx:papers:2312.09081
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2312.09081
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Charles Engel & Kenneth D. West, 2005. "Exchange Rates and Fundamentals," Journal of Political Economy, University of Chicago Press, vol. 113(3), pages 485-517, June.
    2. Barbara Rossi, 2013. "Exchange Rate Predictability," Journal of Economic Literature, American Economic Association, vol. 51(4), pages 1063-1119, December.
    3. Kilian, Lutz & Taylor, Mark P., 2003. "Why is it so difficult to beat the random walk forecast of exchange rates?," Journal of International Economics, Elsevier, vol. 60(1), pages 85-107, May.
    4. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    5. Joscha Beckmann & Gary Koop & Dimitris Korobilis & Rainer Alexander Schüssler, 2020. "Exchange rate predictability and dynamic Bayesian learning," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 410-421, June.
    6. Carlos Bowles & Roberta Friz & Veronique Genre & Geoff Kenny & Aidan Meyler & Tuomas Rautanen, 2007. "The ECB survey of professional forecasters (SPF) – A review after eight years’ experience," Occasional Paper Series 59, European Central Bank.
    7. Pavel Atanasov & Phillip Rescober & Eric Stone & Samuel A. Swift & Emile Servan-Schreiber & Philip Tetlock & Lyle Ungar & Barbara Mellers, 2017. "Distilling the Wisdom of Crowds: Prediction Markets vs. Prediction Polls," Management Science, INFORMS, vol. 63(3), pages 691-706, March.
    8. Olivier Coibion & Yuriy Gorodnichenko, 2015. "Information Rigidity and the Expectations Formation Process: A Simple Framework and New Facts," American Economic Review, American Economic Association, vol. 105(8), pages 2644-2678, August.
    9. Brown, Alasdair & Reade, J. James, 2019. "The wisdom of amateur crowds: Evidence from an online community of sports tipsters," European Journal of Operational Research, Elsevier, vol. 272(3), pages 1073-1081.
    10. Lawrence, Michael & Goodwin, Paul & O'Connor, Marcus & Onkal, Dilek, 2006. "Judgmental forecasting: A review of progress over the last 25 years," International Journal of Forecasting, Elsevier, vol. 22(3), pages 493-518.
    11. Robin Hanson, 2003. "Combinatorial Information Market Design," Information Systems Frontiers, Springer, vol. 5(1), pages 107-119, January.
    12. Satopää, Ville A. & Baron, Jonathan & Foster, Dean P. & Mellers, Barbara A. & Tetlock, Philip E. & Ungar, Lyle H., 2014. "Combining multiple probability predictions using a simple logit model," International Journal of Forecasting, Elsevier, vol. 30(2), pages 344-356.
    13. Meese, Richard A. & Rogoff, Kenneth, 1983. "Empirical exchange rate models of the seventies : Do they fit out of sample?," Journal of International Economics, Elsevier, vol. 14(1-2), pages 3-24, February.
    14. Johannes Leitner & Robert Schmidt, 2006. "A systematic comparison of professional exchange rate forecasts with the judgemental forecasts of novices," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 14(1), pages 87-102, February.
    15. Bo Cowgill & Eric Zitzewitz, 2015. "Corporate Prediction Markets: Evidence from Google, Ford, and Firm X," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 82(4), pages 1309-1341.
    16. Kenny, Geoff & Genre, Véronique & Bowles, Carlos & Friz, Roberta & Meyler, Aidan & Rautanen, Tuomas, 2007. "The ECB survey of professional forecasters (SPF) - A review after eight years' experience," Occasional Paper Series 59, European Central Bank.
    17. repec:reg:rpubli:460 is not listed on IDEAS
    18. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    19. Cornell, W Bradford & Dietrich, J Kimball, 1978. "The Efficiency of the Market for Foreign Exchange under Floating Exchange Rates," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 111-120, February.
    20. Lennart Sjöberg, 2009. "Are all crowds equally wise? a comparison of political election forecasts by experts and the public," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 28(1), pages 1-18.
    21. Berge, Travis J. & Chang, Andrew C. & Sinha, Nitish R., 2019. "Evaluating the conditionality of judgmental forecasts," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1627-1635.
    22. Ian H Giddy & Gunter Dufey, 1975. "The Random Behavior of the Flexible Exchange Rates: Implications for Forecasting," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 6(1), pages 1-32, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    2. Gaglianone, Wagner Piazza & Marins, Jaqueline Terra Moura, 2017. "Evaluation of exchange rate point and density forecasts: An application to Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 707-728.
    3. Kunze, Frederik, 2017. "Predicting exchange rates in Asia: New insights on the accuracy of survey forecasts," University of Göttingen Working Papers in Economics 326, University of Goettingen, Department of Economics.
    4. Demetrescu, Matei & Rodrigues, Paulo M.M. & Taylor, A.M. Robert, 2023. "Transformed regression-based long-horizon predictability tests," Journal of Econometrics, Elsevier, vol. 237(2).
    5. Hai Long Vo & Duc Hong Vo, 2023. "The purchasing power parity and exchange‐rate economics half a century on," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 446-479, April.
    6. Imad Moosa & Kelly Burns, 2016. "The random walk as a forecasting benchmark: drift or no drift?," Applied Economics, Taylor & Francis Journals, vol. 48(43), pages 4131-4142, September.
    7. Byrne, Joseph P. & Korobilis, Dimitris & Ribeiro, Pinho J., 2016. "Exchange rate predictability in a changing world," Journal of International Money and Finance, Elsevier, vol. 62(C), pages 1-24.
    8. Engel, Charles & Wu, Steve Pak Yeung, 2023. "Forecasting the U.S. Dollar in the 21st Century," Journal of International Economics, Elsevier, vol. 141(C).
    9. Hsu, Po-Hsuan & Taylor, Mark P. & Wang, Zigan & Xu, Qi, 2022. "Currency volatility and global technological innovation," Journal of International Economics, Elsevier, vol. 137(C).
    10. Lansing, Kevin J. & Ma, Jun, 2017. "Explaining exchange rate anomalies in a model with Taylor-rule fundamentals and consistent expectations," Journal of International Money and Finance, Elsevier, vol. 70(C), pages 62-87.
    11. Zhang, Hui Jun & Dufour, Jean-Marie & Galbraith, John W., 2016. "Exchange rates and commodity prices: Measuring causality at multiple horizons," Journal of Empirical Finance, Elsevier, vol. 36(C), pages 100-120.
    12. Philip Bertram & Teresa Flock & Jun Ma & Philipp Sibbertsen, 2022. "Real Exchange Rates and Fundamentals in a new Markov‐STAR Model," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 84(2), pages 356-379, April.
    13. Frederik Kunze, 2020. "Predicting exchange rates in Asia: New insights on the accuracy of survey forecasts," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(2), pages 313-333, March.
    14. Beckmann, Joscha & Czudaj, Robert, 2017. "The impact of uncertainty on professional exchange rate forecasts," Journal of International Money and Finance, Elsevier, vol. 73(PB), pages 296-316.
    15. Bekiros, Stelios D., 2014. "Exchange rates and fundamentals: Co-movement, long-run relationships and short-run dynamics," Journal of Banking & Finance, Elsevier, vol. 39(C), pages 117-134.
    16. Meher Manzur, 2018. "Exchange rate economics is always and everywhere controversial," Applied Economics, Taylor & Francis Journals, vol. 50(3), pages 216-232, January.
    17. Wang, Yudong & Liu, Li & Ma, Feng & Diao, Xundi, 2018. "Momentum of return predictability," Journal of Empirical Finance, Elsevier, vol. 45(C), pages 141-156.
    18. Lasha Kavtaradze & Manouchehr Mokhtari, 2018. "Factor Models And Time†Varying Parameter Framework For Forecasting Exchange Rates And Inflation: A Survey," Journal of Economic Surveys, Wiley Blackwell, vol. 32(2), pages 302-334, April.
    19. Agnieszka P. Markiewicz & Ralph C. Verhoeks & Willem F. C. Verschoor & Remco C. J. Zwinkels, 2023. "Inattentive Search for Currency Fundamentals," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(4), pages 907-952, December.
    20. Jens Witkowski & Rupert Freeman & Jennifer Wortman Vaughan & David M. Pennock & Andreas Krause, 2023. "Incentive-Compatible Forecasting Competitions," Management Science, INFORMS, vol. 69(3), pages 1354-1374, March.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2312.09081. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.