Significance Tests Harm Progress in Forecasting

MPRA Paper from University Library of Munich, Germany

Abstract: Based on a summary of prior literature, I conclude that tests of statistical significance harm scientific progress. Efforts to find exceptions to this conclusion have, to date, turned up none. Even when done correctly, significance tests are dangerous. I show that summaries of scientific research do not require tests of statistical significance. I illustrate the dangers of significance tests by examining an application to the M3-Competition. Although the authors of that reanalysis conducted a proper series of statistical tests, they suggest that the original M3 was not justified in concluding that combined forecasts reduce errors and that the selection of the best method is dependent upon the selection of a proper error measure. I show that the original conclusions were justified and that they are correct. Authors should try to avoid tests of statistical significance, journals should discourage them, and readers should ignore them. Instead, to analyze and communicate findings from empirical studies, one should use effect sizes, confidence intervals, replications/extensions, and meta-analyses.

Keywords: accuracy measures; combining forecasts; confidence intervals; effect size; M-competition; meta-analysis; null hypothesis; practical significance; replications (search for similar items in EconPapers)
JEL-codes: C1 C18 (search for similar items in EconPapers)
Date: 2007-04
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (46)

Published in International Journal of Forecasting 23 (2007): pp. 321-336

Downloads: (external link)
https://mpra.ub.uni-muenchen.de/81664/1/MPRA_paper_81664.pdf original version (application/pdf)

Related works:
Journal Article: Significance tests harm progress in forecasting (2007)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:pra:mprapa:81664

Access Statistics for this paper

More papers in MPRA Paper from University Library of Munich, Germany Ludwigstraße 33, D-80539 Munich, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Joachim Winter ().